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Abstract 


This  thesis  research  is  involved  with  the  development  of  new  methodologies  for 

er\hancing  the  experimental  use  of  computer  simulations  to  optimize  predicted  « 

human  performance  in  a  work  domain.  Using  a  computer  simulation  called 

Computer  modeling  Of  Human  Operator  System  Tasks  (CoHOST)  to  test  the 

concepts  in  this  research,  methods  are  developed  that  are  used  to  establish 

confidence  limits  and  significance  thresholds  by  having  the  computer  model  self  • 

report  its  limits.  These  methods,  along  with  experimental  designs  that  are 

tailored  to  the  use  of  computer  simulation  instead  of  human  subject  based 

research,  are  used  in  the  CoHOST  simulation  to  investigate  the  U.S.  Army 

battalion  level  command  and  control  work  domain  during  combat  conditions 

and  develop  recommendations  about  that  domain  based  on  the  experimental 

use  of  CoHOST  with  these  methodologies.  Further,  with  the  realization  that 

analytical  results  showing  strictly  numerical  data  do  not  always  satisfy  the 

need  for  understanding  by  those  who  could  most  benefit  from  the  analysis,  the 

results  are  further  interpreted  in  accordance  with  a  team  performance  model 

and  the  CoHOST  analysis  results  are  mapped  to  it  according  to 

macroergonomic  and  team  performance  concepts. 

The  CoHOST  computer  simulation  models  were  developed  based  on  Army 
needs  stemming  from  the  Persian  Gulf  war.  They  examined  human  mental  and 
physical  performance  capabilities  resulting  from  the  introduction  of  a  new 
command  and  control  vehicle  with  modernized  digital  communications 
systems.  Literature  searches  and  background  investigations  were  conducted, 
and  the  CoHOST  model  architecture  was  developed  that  was  based  on  a 
taxonomy  of  human  performance.  A  computer  simulation  design  was 
implemented  with  these  taxonomic  based  descriptors  of  human  performance  in 
the  military  command  and  control  domain  using  the  commercial  programming 
language  MicroSaintT".  The  original  CoHOST  development  project  developed 
results  that  suggested  that  automation  alone  does  not  necessarily  improve 
human  performance.  ^ 

The  CoHOST  models  were  developed  to  answer  questions  about  whether 

human  operators  could  operate  effectively  in  a  specified  work  domain.  From  ^ 

an  analytical  point  of  view  this  satisfied  queries  being  made  from  the 

developers  of  that  work  domain.  However,  with  these  completed  models 

available,  the  intriguing  possibility  now  exists  to  allow  an  investigation  of  how 

to  optimize  that  work  domain  to  maximize  predicted  human  performance.  By 

developing  an  appropriate  experimental  design  that  allows  evaluative 

conditions  to  be  placed  on  the  simulated  human  operators  in  the  computer 

model  rather  than  live  human  test  subjects,  a  series  of  computer  runs  are  made 

to  establish  test  points  for  identified  dependent  variables  against  specified 
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independent  variables.  With  these  test  points  a  set  of  polynomial  regression 
equations  are  developed  that  describe  the  performance  characteristics 
according  to  these  dependent  variables  of  the  human  operator  in  the  work 
domain  simulated  in  the  model.  The  resulting  regression  equations  are  capable 
of  predicting  any  outcome  the  model  can  produce.  The  optimum  values  for  the 
independent  variables  are  then  determined  that  produce  the  maximum 
predicted  human  performance  according  to  the  dependent  variables. 

The  conclusions  from  the  CoHOST  example  in  this  thesis  complement  the 
results  of  the  original  CoHOST  study  with  the  prediction  that  the  primary 
attentional  focus  of  the  battalion  commander  during  combat  operations  is  on 
establishing  and  maintaining  an  awareness  and  understanding  of  the 
situational  picture  of  the  battlefield  he  is  operating  upon.  Being  able  to  form 
and  sustain  an  accurate  mental  model  of  this  domain  is  the  predicted 
predominant  activity  and  drives  his  ability  to  make  effective  decisions  and 
communicate  those  decisions  to  the  other  members  of  his  team  and  to  elements 
outside  his  team. 

The  potential  specific  benefit  of  this  research  to  the  Army  is  twofold.  First,  the 
research  demonstrates  techniques  and  procedures  that  can  be  used  without  any 
required  modifications  to  the  existing  computer  simulations  that  allow 
significant  predictive  use  to  be  made  of  the  simulation  beyond  its  original 
purpose  and  intent.  Second,  the  use  of  these  techniques  with  CoHOST  is 
developing  conclusions  and  recommendations  from  that  simulation  that  Army 
force  developers  can  use  with  their  continuing  efforts  to  improve  and  enhance 
the  ability  of  commanders  and  other  decision  makers  to  perform  as  new  digital 
communications  systems  and  procedures  are  producing  radical  changes  to  the 
paradigm  that  describes  the  command  and  control  work  domain. 

The  general  benefits  beyond  the  Army  domain  of  this  research  fall  into  the  two 
areas  of  methodological  improvement  of  simulation  based  experimental 
procedures  and  in  the  actual  application  area  of  the  CoHOST  simulation. 
Tailoring  the  experimental  controls  and  development  of  interrogation 
techniques  for  the  self-reporting  and  analysis  of  simulation  parameters  and 
thresholds  are  topics  that  bode  for  future  study.  The  CoHOST  simulation, 
while  used  in  this  thesis  as  an  example  of  new  and  tailored  techniques  for 
computer  simulation  based  research,  has  nevertheless  produced  conclusions 
that  deviate  somewhat  from  prevailing  thought  in  military  command  and 
control.  Refinement  of  this  simulation  and  its  use  in  an  even  more  thorough 
simulation  based  study  could  further  address  whether  the  military  decision 
making  process  itself  or  contributing  factors  such  as  development  of  mental 
models  for  understanding  of  the  situation  is  or  should  be  the  primary  focus  of 
team  decision  makers  in  the  military  command  and  control  domain. 
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(Abstract) 

This  thesis  research  is  involved  with  the  development  of  new  methodologies  for 
enhancing  the  experimental  use  of  computer  simulations  to  optimize  predicted  hiunan 
performance  in  a  work  domain.  Using  a  computer  simulation  called  Computer  modeling  Of 
Human  Operator  System  Tasks  (CoHOST)  to  test  the  concepts  in  this  research,  methods  are 
developed  that  are  used  to  establish  confidence  limits  and  significance  thresholds  by  having  the 
computer  model  self  report  its  limits.  These  methods,  along  with  experimental  designs  that  are 
tailored  to  the  use  of  computer  simulation  instead  of  human  subject  based  research,  are  used  in 
the  CoHOST  simulation  to  investigate  the  U.S.  Army  battalion  level  command  and  control  work 
domain  during  combat  conditions  and  develop  recommendations  about  that  domain  based  on  the 
experimental  use  of  CoHOST  with  these  methodologies.  Further,  with  the  realization  that 
analytical  results  showing  strictly  niunerical  data  do  not  always  satisfy  the  need  for 
understanding  by  diose  who  could  most  benefit  from  the  analysis,  the  results  are  further 
interpreted  in  accordance  with  a  team  performance  model  and  the  CoHOST  analysis  results  are 
mapped  to  it  according  to  macroergonomic  and  team  performance  concepts. 

The  CoHOST  computer  simulation  models  were  developed  based  on  Army  needs 
stemming  from  the  Persian  Gulf  war.  They  examined  human  mental  and  physical  performance 
capabilities  resulting  from  the  introduction  of  a  new  command  and  control  vehicle  with 
modernized  digital  commimications  systems.  Literature  searches  and  background  investigations 
were  conducted,  and  the  CoHOST  model  architecture  was  developed  that  was  based  on  a 
taxonomy  of  human  performance.  A  computer  simulation  design  was  implemented  with  these 
taxonomic  based  descriptors  of  human  performance  in  the  military  command  and  control  domain 
using  the  commercial  programming  language  MicroSaint™.  The  original  CoHOST  development 
project  developed  residts  that  suggested  that  automation  alone  does  not  necessarily  improve 
human  performance. 

The  CoHOST  models  were  developed  to  answer  questions  about  whether  human 
operators  could  operate  effectively  in  a  specified  work  domain.  From  an  analytical  point  of  view 
tMs  satisfied  queries  being  made  from  the  developers  of  that  work  domain.  However,  with  these 
completed  models  available,  the  intriguing  possibility  now  exists  to  allow  an  investigation  of 
how  to  optimize  that  woric  domain  to  maximize  predicted  human  performance.  By  developing 
an  appropriate  experimental  design  that  allows  evaluative  conditions  to  be  placed  on  the 
simulated  human  operators  in  the  computer  model  rather  than  live  human  test  subjects,  a  series 
of  computer  runs  are  made  to  establish  test  points  for  identified  dependent  variables  against 
specified  independent  variables.  With  these  test  points  a  set  of  polynomial  regression  equations 
are  developed  that  describe  the  performance  characteristics  according  to  these  dependent 
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variables  of  the  human  operator  in  the  work  domain  simulated  in  the  model.  The  resulting 
regression  equations  are  capable  of  predicting  any  outcome  the  model  can  produce.  The  optimum 
values  for  the  independent  variables  are  then  determined  that  produce  the  maximum  predicted 
human  performance  according  to  the  dependent  variables. 

The  conclusions  fix)m  the  CoHOST  example  in  this  thesis  complement  the  results  of  the 
original  CoHOST  study  with  the  prediction  that  Ae  primary  attentional  focus  of  the  battalion 
commander  during  combat  operations  is  on  establishing  and  maintaining  an  awareness  and 
understanding  of  the  situational  picture  of  the  battlefield  he  is  operating  upon.  Being  able  to 
form  and  sustain  an  accurate  mental  model  of  this  domain  is  the  predicted  predominant  activity 
and  drives  his  ability  to  make  effective  decisions  and  communicate  those  decisions  to  the  other 
members  of  his  team  and  to  elements  outside  his  team. 

The  potential  specific  benefit  of  this  research  to  the  Army  is  twofold.  First,  the  research 
demonstrates  techniques  and  procedures  that  can  be  used  without  any  required  modifications  to 
the  existing  computer  simulations  that  allow  significant  predictive  use  to  be  made  of  the 
simulation  beyond  its  original  purpose  and  intent.  Second,  the  use  of  these  techniques  with 
CoHOST  is  developing  conclusions  and  recommendations  from  that  simulation  that  Army  force 
developers  can  use  with  their  continuing  efforts  to  improve  and  enhance  the  ability  of 
commanders  and  other  decision  makers  to  perform  as  new  digital  communications  systems  and 
procedures  are  producing  radical  changes  to  the  paradigm  that  describes  the  command  and 
control  work  domain. 

The  general  benefits  beyond  the  Army  domain  of  this  research  fall  into  the  two  areas  of 
methodological  improvement  of  simulation  based  experimental  procedures  and  in  the  actual 
application  area  of  the  CoHOST  simulation.  Tailoring  the  experimental  controls  and 
development  of  interrogation  techniques  for  the  self-reporting  and  analysis  of  simulation 
parameters  and  thresholds  are  topics  that  bode  for  future  study.  The  CoHOST  simulation,  while 
used  in  this  thesis  as  an  example  of  new  and  tailored  techniques  for  computer  simulation  based 
research,  has  nevertheless  produced  conclusions  that  deviate  somewhat  from  prevailing  thought 
in  military  command  and  control.  Refinement  of  this  simulation  and  its  use  in  an  even  more 
thorough  simulation  based  study  could  further  address  whether  the  military  decision  making 
process  itself  or  contributing  factors  such  as  development  of  mental  models  for  understanding  of 
the  situation  is  or  should  be  the  primary  focus  of  team  decision  makers  in  the  military  command 
and  control  domain. 
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1.  Introduction. 


Following  the  Persian  Gulf  War,  when  the  U.S.  Army  determined  that  its  current  armored 
command  and  control  vehicle  was  obsolete,  the  Human  Research  and  Engineering  Directorate  of 
^  the  U.S.  Army  Research  Laboratory  developed  a  series  of  computer  simulation  models  to 

examine  human  mental  and  physical  performance  capabilities  resulting  from  the  introduction  of 
a  new  vehicle  with  modernized  digital  communications  systems.  A  computer  simulation  design 
was  implemented  with  taxonomic  based  descriptors  of  human  performance  in  the  military 
command  and  control  domain  using  the  commercial  programming  language  MicroSaint™.  A 
series  of  computer  models  called  Computer  modeling  Of  Human  Operator  System  Tasks 
(CoHOST)  (Middlebrooks  et  al.,  1999)  was  written  and  results  were  developed  that  addressed 
questions  being  posed  by  the  developers  of  the  new  vehicle. 

With  these  completed  models  on  hand,  the  intriguing  possibility  exists  to  carry  the 
research  beyond  the  purpose  of  the  origirud  project  and  allow  an  investigation  of  how  to  optimize 
the  simulated  work  domain  to  maximize  predicted  human  performance.  This  thesis  uses  an 
experimental  design  to  evaluate  performance  conditions  to  be  placed  on  the  simulated  human 
operators  in  the  computer  model.  A  series  of  computer  runs  is  then  conducted  to  establish  test 
points  for  identified  dependent  variables  against  specified  independent  variables.  With  these  test 
points  a  set  of  polynomial  regression  equations  is  developed  that  describe  the  performance 
characteristics  according  to  these  dependent  variables  of  the  human  operator  in  the  work  domain 
simulated  in  the  model.  The  resulting  regression  equations  are  predictive  of  any  outcome  the 
model  is  capable  of  producing.  Finally,  the  derivative  of  the  equations  is  taken  and  set  equal  to 
zero  to  provide  the  optimum  values  for  the  indqiendent  variables  that  will  produce  the  maximum 
human  performance  according  to  the  dependent  variables. 

1.1.  Review  of  the  Literature. 

The  use  of  computer  simulation  as  an  exploratory  tool  to  support  experimental  design  is 
well  docmnented  in  the  literature.  It  can  be  thought  of  as  a  merging  of  the  concepts  of  system 
simulation,  system  modeling  and  the  use  of  the  digital  computer  as  a  research  aid  (Whicker  and 
Sigelman,  1991b).  Five  elements  of  a  computer  simulation  are  described  as: 

•  Assumptions  in  the  simulation. 

•  Parameters  or  fixed  values  providing  input  to  the  simulation. 
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•  Independent  variables  providing  input  to  the  simulation. 

•  Algorithms  embedded  in  the  simulation. 

•  Simulation  output  contained  in  the  dependent  variables. 

The  steps  that  comprise  the  use  of  a  con^uter  simulation  in  a  simulation  based  study  include 
(Banks,  Carson,  and  Nelson,  1996): 

•  Formulate  the  problem. 

•  Set  objectives  and  develop  a  project  plan. 

•  Conceptualize  or  design  the  model. 

•  Collect  supporting  data. 

•  Translate  the  model  design  into  a  computer  simulation. 

•  Verify  the  simulation. 

•  Validate  the  simulation. 

•  Develop  an  experimental  design  based  on  the  simulation. 

•  Perform  simulation  runs  to  gather  data  and  then  analyze  it. 

•  Determine  if  additional  runs  are  required. 

•  Report  and  document  the  results. 

•  Implement  the  results. 

Other  interpretations  are  applied  to  this  process  but  the  concept  is  basically  the  same  (Law  and 
Kelton,  2000): 

•  Problem  formulation  and  study  planning. 

•  Data  collection  and  model  definition. 

•  Validating  the  conceptual  model. 

•  Write  the  computer  program.  Verify  it. 

•  Perform  pilot  runs  of  the  simulation. 

•  Validate  the  programmed  model. 

•  Develop  an  experimental  design  based  on  the  model. 

•  Perform  simulation  runs  to  gather  data. 

•  Analyze  the  simulation  data. 


Report,  present,  and  employ  the  results. 


A  simulation  is  described  as  a  numerical  technique  for  conducting  experiments  on  mathematical 

^  and  logical  models  that  describe  the  behavior  of  a  system  using  a  digital  computer  (Naylor  and 

r 

Gianturco,  1966).  Naylor’s  process  definition  includes: 

•  Problem  formulation. 

*  •  Data  collection. 

•  Model  formulation. 

•  Parameter  estimation. 

•  Evaluation  of  the  model  and  parameter  estimates. 

•  Formulation  of  a  computer  program. 

•  Validation. 

•  Experimental  design. 

•  Analysis  of  simulated  data. 

If  this  process  has  been  performed  to  some  stage  of  completion  and  a  completed 
simulation  model  is  available  for  use,  then  the  real  benefit  of  having  a  simulation  model  comes 
into  existence.  One  use  of  such  a  model  is  the  investigation  of  what  combination  of  input 
variables  will  provide  the  optimal  mix  of  the  output  variables  (Clayton,  Weber,  and  Taylor, 
1982).  Several  approaches  can  be  used  to  achieve  this  result.  The  first  is  a  brute  force  approach 
whereby  all  possible  combinations  of  the  model  inputs  are  evaluated  for  their  output.  When 
using  factorial  experimentation  the  combinations  can  become  unmanageable  very  quickly  even 
with  the  aid  of  the  computer  running  simulated  experiments.  The  second  approach  is  to  run 
^  selected  combinations  of  input  variables  selected  according  to  an  experimental  design  and  then 

use  regression  analysis  to  estimate  the  equations  that  will  produce  the  output  values.  A  third 
approach  might  be  to  use  what  is  called  a  ‘direct  search  procedure’  that  considers  different 
«  objectives  that  does  not  require  the  knowledge  of  exact  model  equations. 

The  second  approach  described  above  can  be  employed  in  a  manner  where  the  simulation 
itself  is  simply  considered  to  be  a  ‘black  box’  where  some  translation  of  the  input  conditions  is 
manipulated  to  produce  a  resulting  set  of  output  conditions  (Smith,  1973;  Smith  and  Mauro, 
1981).  With  this  method,  one  approach  is  to  use  a  factorial  design  to  evaluate  the  output  for  all 
combinations  of  the  input  factors.  Then  either  select  the  input  combination  that  produces  the 
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best  results  or  a  set  of  points  in  a  region  and  use  them  to  fit  a  regression  equation.  This  equation 
can  then  be  used  to  predict  the  optimum  values  of  the  outputs. 

Stasser  (1990)  noted  that  the  use  of  computer  simulation  in  social  science  research 
^  declined  after  an  initial  interest  in  the  late  1 960’s.  One  reason  that  was  postulated  was  that  it  is 

possible  that  simulations  of  social  behavior  can  yield  patterns  of  discrete  and  qualitative  results 
that  cannot  easily  be  represented  in  statistical  testing.  It  could  be  further  stated  that  computer 

*  modeling  of  socio-technical  inteipersonal  tasks  is  not  an  exact  science.  As  an  exploratory  tool, 
simulation  can  be  very  useful  for  planning  empirical  research  and  for  obtaining  tentative  answers 
to  selected  questions.  It  still  requires  good  theorizing  with  intelligence,  creativity  and  style  from 
human  theorists  to  make  sense  from  the  results  of  the  simulation  (Stasser,  1990).  The  primary 
purpose  of  a  computer  simulation  supported  experiment  is  to  fmd  a  simple  but  accurate  function 
that  represents,  over  the  region  of  interest,  the  true  function  comprising  the  computer  model  itself 
(Webb,  1970).  While  these  references  are  from  ten  to  thirty  years  ago  they,  nevertheless, 
illustrate  a  continuing  lack  of  the  use  of  computer  simulations  in  support  of  empirical  inquiry 
that  has  continued  to  exist.  The  purpose  of  this  thesis  is  to  address  this  shortcoming  and 
demonstrate  how  well  established  procedures  can  be  tailored  to  this  type  of  research. 

Specific  areas  of  interest  to  this  thesis  topic  that  will  be  explored  in  the  literature  are  the 
use  of  simulations  with  experimental  designs,  the  use  of  simulations  to  describe  human 
performance,  and  the  use  of  regression  analysis  with  conq)uter  simulations. 

1.1.1.  Use  of  Simulations  With  Experimental  Designs. 

The  basic  nature  and  process  of  interrogating  simulation  models  according  to  a  formal 
experimental  design  has  been  described  and  is  well  documented  (Banks  et  al,  1996)  (Law  and 
g  Kelton,  2000).  The  use  of  these  techniques  in  social  science  research  apparently  is  not  extensive 

(Stasser,  1990),  but  there  is  a  continuing  and  significant  effort  being  applied  to  the  use  of  ftiese 
principles  both  in  industry  and  in  academia. 

*  For  example,  one  experiment  that  used  a  replication  count  of  1 00,000  simulated  4 
treatment  groups  of  16  subjects  each  that  converted  randomly  drawn  pairs  from  a  uniform 
distribution  into  random  normal  deviates  to  evaluate  the  differences  between  Type  I  error  rates 
for  a  variety  of  multiple  comparisons  (Klockars  and  Hancock,  1994).  Another  example  used 
computer  simulation  to  investigate  problems  connected  with  the  genetic  analysis  of  continuously 
variable  behavioral  patterns  in  human  populations  (Eaves,  1972).  Environmental  variation  in 
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human  populations  has  been  investigated  (Eaves,  1970),  and  interaction  effects  using  constructed 
populations  to  investigate  different  hypotheses  has  been  performed  (Gabrielsson  and  Seeger, 
1971).  Techniques  to  deal  with  such  issues  as  initialization  bias  of  the  independent  variables  in 
^  nonterminating  simulations  has  been  investigated  and  described  (Schruben  and  Goldsman, 

1984).  One  thesis  that  investigated  predicted  simulated  human  performance  using  empirically 
derived  input  data  employed  the  commercial  software  package  SAINT^^  (System  Analysis  of 

*  Integrated  Networks  of  Tasks)  which  was  a  predecessor  to  the  MicroSAINT^*'  package  in  current 
use  (Askren,  1976).  In  this  case,  the  empirically  derived  data  was  manipulated  into  a  set  of 
linear  regression  equations  that  were  used  to  provide  input  to  the  computer  model.  Another 
thesis  using  a  commercial  simulation  package  used  the  SIMSCRIPT  n.5  discrete  event 
simulation  language  to  model  the  U.S.  Army  Coips  of  Engineer’s  construction  change  order 
processing  procedure.  This  application  included  the  use  of  the  Ramberg-Schmeiser  percentile 
probability  distribution  function  and  a  regression  analysis  to  model  the  network  activities  (Curtis, 
1986).  In  another  case,  three  computer  simulation  sessions  investigated  the  impact  of  two 
response  styles  against  the  validity  of  results  obtained  from  regression  and  factor  analysis.  This 
simulation  study  approach  over  a  formal  experimental  design  using  human  test  subjects  was 
selected  to  allow  complete  control  over  the  manipulations,  to  enable  the  ability  to  vary  the 
number  of  subjects  in  the  test  conditions,  and  to  account  for  difficulties  of  analytical  approaches 
when  two  response  variables  are  combined  (Heide  and  Gronhaug,  1992). 

1.1.2.  Use  of  Simulations  to  Describe  Human  Performance. 

Early  evolutions  of  the  human  computer  interface  have  been  described  in  terms  of  19S0*s 
vintage  technology  as  being  much  ‘simpler’  but  far  less  effective  (Muckier,  1987).  This 
g  observation,  made  in  the  mid  1980’s,  commented  that  the  technology  was  in  a  transition  phase 

where  nothing  worked  very  well  but  the  possibilities  were  exciting.  Whether  or  not  the  truth  of 
this  observation  has  changed  in  the  intervening  20  years  is  open  to  speculation,  however,  there 

*  are  tools  and  disciplines  that  have  emerged  that  directly  address  the  nature  of  the  problem. 

The  discipline  of  human  computer  interaction  (HCI)  has  become  a  recognized  field  of 
study  to  investigate  the  unique  nature  of  the  interactions  of  a  human  system  interface  where  the 
system  is  a  computer  or  is  computer  based  or  driven.  Investigations  into  this  field  discuss  the 
concept  of  using  models  of  the  user  as  an  interface  design  tool  (Williges,  1987).  Because  of  the 
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nature  of  HCI  many  times  the  computer  is  used  to  support  experimental  design  investigations 
using  the  actual  computer  based  system  itself.  This  is  not  simulation  based  experimentation  but 
rather  testing  where  the  system  itself  exists  on  a  computer.  An  example  of  this  was  a  study 
(Cohill  and  Williges,  1985)  that  used  a  2^  between  subjects  design  to  look  at  different  forms  of 
the  HELP  function  on  the  computer.  The  actual  simulation  of  the  human  component  in  an  HCI 
system  can  become  complicated  and  subject  to  review  and  introspection.  A  simulation  of  the 
single  operator  in  a  semiautomatic  radar  surveillance  system  using  a  time  compressed  real-time 
cathode  ray  tube  display  was  conducted  (Mills  and  Williges,  1973),  and  was  followed  by  an 
assessment  of  the  validity  of  the  empirically  derived  prediction  equations  of  the  operator 
performance  in  this  simulated  system  (Williges  and  Mills,  1973). 

The  concept  of  cognitive  compatibility  has  been  identified  as  a  central  component  in  the 
field  of  HCI  and  is  a  useful  concept  to  account  for  user’s  behavior  in  this  field  (Streitz,  1987). 
Investigations  of  concepts  such  as  this  can  be  performed  empirically  with  live  hiiman  test 
subjects,  however,  the  complex  nature  of  the  interactions  involved  are  difficult  to  reproduce  with 
subjects  that  may  be  available  for  only  one  or  a  few  test  sessions.  Computer  simulation  provides 
the  repeatability  and  manipulation  ability  to  replicate  many  different  test  conditions  with 
simulated  test  subjects  that  obediently  perform  as  directed.  Of  course,  the  disadvantage  of 
simulated  test  subjects  is  that  they  are  only  as  cognitively  complex  as  their  programming  allows 
and  are  typically  suitable  only  for  the  test  domain  of  the  current  simulation. 

HCI  has  emerged  as  a  research  field  because  of  the  exploding  technology  that  spawned  it. 
Technology  drives  HCI  and  the  most  vital  part  in  HCI  design  is  the  generation  and  utilization  of 
basic  generic  research  devoted  to  it  (Salvendy,  1987).  Using  this  technology  to  evolve  the  tools 
needed  to  study  and  improve  it,  such  as  with  computer  simulations,  will  enable  positive 
outcomes  to  predictions  from  technical  theorists  such  as  Muckier  and  others. 

1.1.3.  Use  of  Regression  Analysis  With  Computer  Simulations. 

It  has  already  been  observed  that  regression  analysis  can  be  a  useful  tool  for  taking  the 
results  of  computer  simulation  and  determining  what  the  optimum  value  of  the  dependent 
variables  might  be  (Smith,  1973).  One  example  of  this  process  was  where  data  fix>m  a 
deterministic  simulation  model  that  was  used  to  predict  statistically  noisy  experimental  data 
simulating  the  psychological  processes  in  a  language  processing  simulation  was  compared  with 
single  sentence  reading  times  by  fitting  a  linear  equation  to  the  reading  times  (Kieras,  1979). 
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Another  example  involved  the  use  of  a  computer  simulation  to  investigate  moderated  multiple 
regression  techniques  for  moderating  variables  in  industrial  /  organizational  psychology  research 
(Paunonen  and  Jackson,  1988).  One  simulation  study  looked  at  the  tool  itself  by  investigating 
the  relative  power  of  different  moderated  multiple  regression  techniques  with  or  without  a 
dichotomized  moderator  and  correlational  analysis  (Mason,  Tu,  and  Cauce,  1996). 

The  ability  to  take  output  from  an  experimental  design  study  that  has  been  generated 
either  with  or  without  a  computer  and  apply  algebraic  manipulations  to  representations  of  that 
data  to  determine  maximum  and  /  or  minimum  performance  limits  can  be  a  powerful  technique 
for  describing  the  performance  domain. 

1 .2.  Research  Goals  and  Objectives. 

The  purpose  of  this  thesis  is  twofold.  First,  and  foremost,  this  project  is  to  develop 
methodologies  that  expand  the  ability  to  use  computer  simulations  to  support  human 
performance  studies  using  experimental  designs.  Discussions  of  research  implications  and 
methodological  procedures  are  conducted  to  explore  these  issues  and  to  provide  proposals  and 
suggestions  for  the  conduct  of  computer  simulation  based  experiments  along  with  proposed 
guidelines  for  the  efficient  conduct  of  such  experiments.  Secondly,  this  project  uses  these 
methods  in  a  study  that  investigates  an  actual  problem  the  U.S.  Army  is  working  to  resolve.  This 
issue  is  how  to  optimize  human  performance  efficiency  in  a  military  conunand  and  control  work 
team  during  combat  operations. 

1.2.1.  Optimizing  Experimental  Designs  Of  Human  Performance  Using  Computer  Simulation. 

The  first  step  is  to  determine  an  experimental  design  appropriate  for  the  investigation. 
While  this  statement  may  seem  obvious,  the  normal  conq>romises  between  design  efficiency  and 
analysis  resolution  desires  require  special  consideration  when  the  data  is  to  be  generated  by  a 
computer  simulation.  Current  computer  technology  that  allows  the  generation  of  large  amounts 
of  data  in  a  relatively  short  time  can  foster  the  belief  that  designs  that  would  be  too  large  and 
complex  to  collect  data  upon  in  actual  empirical  study  can  be  performed  in  a  straightforward 
manner  on  the  computer.  While  this  might  be  true  to  a  certain  extent,  the  researcher  nevertheless 
must  exercise  prudence  in  the  selection  of  resolution  and  treatment  levels  for  the  study. 

Computer  runs  that  produce  Resolution  V  results,  for  example,  produce  results  that  are  of  no  use 
if  the  data  can  only  be  analyzed  and  interpreted  at  the  Resolution  HI  level.  The  temptation  to 
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gather  as  much  as  possible  and  then  sort  it  out  later  should  not  be  attempted  just  because  the 
computer  allows  it  to  happen.  A  well  thought  out  and  efficient  design  that  gathers  just  the  data 
needed  is  just  as  important  with  the  computer  as  it  is  with  empirical  study. 

^  On  the  other  hand,  the  use  of  the  computer  can  make  possible  research  attempts  that 

would  not  be  possible  with  human  study  participants  because  of  considerations  such  as  the 
number  of  factors  in  the  design,  unavailability  or  nonexistence  of  experimental  apparatus,  or 
t  other  factors  that  would  make  the  research  too  complex  or  too  costly  to  perform.  In  cases  like 

this,  excess  complexity  in  the  experimental  design  can  push  the  computer  beyond  its  limits  to 
perform  in  the  time  required  to  run  the  simulation  and  perform  the  number  of  replication  runs 
required.  In  this  situation,  just  as  in  conventional  empirical  work,  an  efficient  design  may  make 
possible  work  that  could  not  be  performed  for  a  full  factorial  experiment  or  in  cases  where  the 
design  is  more  complex  than  normal.  Designs  that  have  a  number  of  factors  of  six  or  more  and 
increasing  numbers  of  treatment  levels  may  not  be  realistically  possible  if  it  takes  the  computer 
hours  to  perform  one  replication  run.  In  this  case  the  design  must  be  constrained  to  levels  that 
are  realistically  possible  with  the  computer.  The  importance,  however,  of  determining  the  level 
of  resolution  possible  in  the  analysis  from  the  number  of  treatments  in  the  data  must  ensure  that 
the  efficient  design  that  is  started  with  will  enable  the  desired  analyses  after  the  data  is  collected. 
If  it  is  determined  that  the  initial  design  cannot  support  the  desired  or  required  level  of  analysis 
then  other  techniques  such  as  sequential  experimentation  (Han,  Williges,  and  Williges,  1997; 
Williges,  Williges,  and  Han,  1992;  Williges,  Williges,  and  Han,  1993)  should  be  explored  and 
are  just  as  appropriate  wifti  the  computer  as  with  conventional  experimentation. 

After  weighing  tire  capabilities  and  requirements  of  the  simulation,  the  task  is  then  to 
choose  the  basic  design.  The  best  solution  is  to  just  select  a  full  factorial  design.  However,  this 
*  may  not  be  possible  as  previously  discussed.  Reasons  that  may  preclude  it  are  large  numbers  of 

experimental  factors  and  time  constraints  on  the  available  computer  resources.  The  next  choice 
^  is  logically  a  fractional  factorial  design,  a  central  composite  design  (CCD)  or  some  combination 

of  the  two.  If  the  variables  are  quantitative  and  there  are  only  two  treatment  levels  for  each 
variable  then  the  central  composite  design  can  provide  good  efficiency.  However,  the 
requirement  for  the  variables  to  be  quantitative  limits  the  applicability  of  the  CCD.  Also,  with 
only  two  treatment  levels  per  variable  it  is  only  possible  to  examine  linear  components  of  the 
main  effects  in  the  output  analysis.  A  fractional  factorial  design  can  overcome  these  limitations. 
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Typically,  in  human  factors  work  the  ability  to  interpret  more  than  two-way  interactions  is  not 
possible.  A  Resolution  V  experiment  that  resolves  all  two  way  interactions  is  achievable  with 
many  fractional  factorial  combinations.  In  some  cases  only  the  main  effects  can  be  interpreted 
^  and  a  Resolution  m  experiment  that  provides  this  is  even  more  achievable  with  fractional 

factorial  techniques.  In  fractional  factorial  designs,  as  with  CCD’s,  the  number  of  treatment 
levels  also  becomes  an  issue.  If  there  are  only  two  treatment  levels  then  the  ability  to  resolve 
^  higher  order  components  in  the  output  again  is  an  issue.  One  possibility  to  overcome  this 

limitation  is  to  augment  the  2"  design  with  one  more  treatment  level.  In  many  cases  where  there 
are  only  two  treatment  levels  the  ability  to  define  a  third  center  point  treatment  is  a  possibility. 
This  type  of  design  can  be  called  an  augmented  design  where  the  fractional  factorial  design  with 
two  treatment  levels  is  augmented  with  one  more  treatment  from  the  center  point.  This 
distinction  is  made  as  compared  to  expanding  the  number  of  treatments  to  a  full  3”  fractional 
factorial  design  which  would  require  that  more  treatments  be  conducted. 

A  final  consideration  in  tailoring  an  experimental  design  for  use  with  computer 
simulations  is  to  consider  unbalancing  the  treatment  orders.  Instead  of  having  a  set  munber,  for 
example  3  or  5,  of  simulated  subjects  per  treatment  condition,  a  possibility  is  to  reduce  this 
number  to  only  one.  Then,  if  the  design  is  augmented  with  another  treatment  level  such  as  a 
center  point  treatment,  pick  a  number  of  repetitions  for  this  treatment  that  will  keep  the  error 
term  for  the  F  ratio  within  an  acceptable  limit.  The  advantage  in  doing  this  is  to  reduce  the 
number  of  required  computer  simulation  runs  to  support  the  experimental  design.  The 
disadvantage  is  that  error  due  to  random  variability  in  the  computer  simulation  could  be 
magnified  to  observable  levels  because  the  number  of  simulated  subjects  per  treatment  condition 
may  not  be  enough  to  average  out  the  effects. 

There  are  also  considerations  to  be  made  about  the  use  of  the  simulation  itself  The  first 
is  the  well  docmnented  (Banks  et  al.,  1996)  procedure  of  using  multiple  replications  of  the 
computer  simulation  to  hold  the  error  limit  relative  to  the  mean  to  within  an  acceptable  limit. 

* 

This  involves  making  multiple  runs  (replications)  of  the  simulation  and  then  averaging  the 
results  for  each  output  variable  to  hold  the  results  from  the  random  variability  to  the  desired 
limit.  Alternatively,  the  results  from  each  replication  run  can  be  treated  as  an  individual 
treatment  to  be  examined  by  the  statistical  analysis. 


9 


A  technique  that  is  imique  to  simulation  based  experimentation  involves  the  use  of 
dummy  independent  variables.  This  procedure  evolves  from  the  difficulty  in  determining  the 
appropriate  level  of  significance  to  use  in  interpreting  P  values  from  statistical  run  results  for  the 
^  dependent  variables.  While  the  commonly  used  values  of  .05,  .01,  or  .001  can  be  used  in  a 

manual  table  lookup,  a  dynamic  alternative  is  to  query  the  model  itself  to  establish  the  actual 
level  of  significance  for  random  variability  in  the  simulation.  The  technique  involves  the 
f  inclusion  of  another  independent  variable  in  the  experimental  design  that  is  not  recognized  or 

used  in  the  simulation.  Treatment  runs  are  made  that  include  this  dummy  variable  using  preset 
treatment  values  that  do  not  change  from  treatment  to  treatment  and  the  output  is  analyzed  for  it. 
The  P  value  for  this  variable  then  represents  random  or  unmanipulated  variability  in  the 
computer  simulation.  Using  this  value  for  the  threshold  cutoff  for  significance  as  opposed  to  one 
of  the  traditional  levels,  all  the  other  variables  are  compared  to  it  and  those  that  are  less  than  this 
threshold  are  deemed  significant  and  those  that  are  higher  are  not  significant.  The  point  of  note 
is  that  this  is  a  dynamically  determined  significance  threshold  for  this  simulation  and  a  much 
tighter  fit  for  regression  equations  can  be  determined  and  closer  predictions  for  ANOVA 
calculations  can  be  made  using  it. 

The  final  component  of  this  methodology  and  resulting  series  of  guidelines  includes  a 
process  many  times  omitted  or  neglected  in  conventional  experimental  studies.  This  component 
is  the  requirement  to  explain  and  interpret  the  results  of  statistical  and  experimental  conclusions 
into  terms  that  occupants  and  developers  of  the  work  domain  in  question  can  understand.  In 
order  to  do  this  the  evolving  human  factors  sub  discipline  of  macroergonomics  along  with  team 
performance  theory  is  invoked  to  link  experimental  results  to  understandable  descriptions.  The 
goal  is  to  pick  a  viable  and  descriptive  model  from  the  literature  that  will  provide  the  basis  for  a 
descriptive  imderstanding  of  the  CoHOST  predictive  implications. 

1 .2.2.  A  Simulation  Example  -  U.S.  Army  Battalion  Command  And  Control  Team. 

*  The  methodology  that  has  been  described  needs  to  be  implemented  in  an  actual  study  to 

prove  its  efficacy.  Several  factors  should  be  considered  in  the  selection  of  an  appropriate 
simulation  to  use  for  experimental  investigations.  Regardless  of  whether  a  new  or  existing 
simulation  is  to  be  used,  several  questions  must  be  addressed  before  it  can  be  considered  to  be 
viable  for  the  experiment.  These  questions  include: 

•  What  is  the  real  world  enviromnent  or  scenario? 


10 


Hi 


•  What  is  the  level  of  accuracy  or  replication  in  the  simulation  of  the  real  world? 

•  How  much  realism  is  good  enough? 

Since  no  simulation  is  likely  to  ever  completely  replicate  the  conditions  of  the  real  world  the 
simulation  must  be  accepted  for  the  level  of  realism  existing  in  it.  This  acceptance  of  risk 
acknowledges  the  level  of  realism  in  the  simulation  and  provides  an  implied  willingness  to 
accept  the  predictions  from  the  simulation  to  support  the  real  project  being  examined.  These 
#  decisions  for  model  acceptance  can  be  enhanced  through  the  use  of  subject  matter  experts  (SME) 

that  can  explicitly  describe  what  the  real  world  scenario  looks  like  and  by  technical  experts  who 
can  describe  the  level  of  authenticity  in  the  simulation  algorithms  and  constructs  but  it  is  the 
experiment  designer  who  must  decide  how  much  is  good  enough.  Some  of  the  reasons  that  the 
use  of  a  computer  simulation  becomes  viable  over  real  world  experimentation  evolves  from 
reasons  as  to  why  the  real  system  cannot  be  fully  exercised  or  tested.  The  system  may  be: 

■  Too  costly. 

■  Too  dangerous. 

■  Too  timely  (i.e.,  requires  too  much  time  to  test). 

The  simulation  may  also  be  desired  for  its  ability  to  train  operators  and  controllers  in  a  more  cost 
effective  or  time  effective  manner. 

For  this  thesis  it  is  desired  to  use  a  simulation  already  in  existence.  There  are  a  couple  of 
reasons  for  this  desire.  First,  this  effort  is  not  a  simulation  development  effort  but  rather  an 
experimental  design  and  analysis  effort.  Second,  in  the  government  as  in  industry,  there  are 
countless  simulations  that  get  developed  and  used  for  a  single  purpose  and  then  are  shelved  or 
discarded  notwithstanding  development  costs  that  might  be  substantial  in  terms  of  money,  time, 
and  effort.  A  significant  objective  of  this  thesis  is  to  demonstrate  that  a  preexisting  simulation 
^  can  be  taken  and  used  unmodified  to  support  ‘what-if  types  of  studies  that  have  the  potential  to 

far  transcend  the  original  purpose  of  the  simulation.  By  taking  such  a  simulation  and 
manipulating  its  data  according  to  an  experimental  design,  the  desire  is  to  show  that  meaningful 
use  can  be  made  of  it  that  may  even  outweigh  the  benefits  from  it’s  original  use. 

The  computer  simulation  chosen  to  demonstrate  this  process  is  titled  Con:q)uter  modeling 
of  Human  Operator  System  Tasks  (CoHOST)  developed  by  the  U.S.  Army  Research  Laboratory. 
This  was  part  of  a  multi-year,  multi-million  dollar  effort  to  investigate  whether  human  operators 
could  operate  more  efficiently  in  a  proposed  new  work  domain  characterized  by  next  generation 
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digital  communication  systems  inside  a  moving  vehicle  under  combat  conditions.  After  an 
investment  of  several  million  dollars  over  several  years  that  primarily  consisted  of  labor  related 
costs  for  the  project  team,  a  series  of  CoHOST  models  were  developed  that  made  predictions  for 
the  answers  to  questions  originally  posed  by  the  developers  of  the  new  work  domain.  After  the 
final  delivery  of  the  CoHOST  project  data  to  the  clients  the  project  was  terminated  and  the 
project  team  reassigned.  As  a  fully  operational  computer  simulation  that  was  the  product  of  an 
extensive  design,  development,  programming  and  V&V  (verification  and  validation)  effort, 
coupled  with  the  author’s  familiarity  with  and  access  to  the  simulation,  CoHOST  proved  to  be  an 
excellent  choice  for  this  project.  The  fact  that  the  original  development  project  was  over  and  the 
simulation  existed  in  a  static  form  not  being  constantly  changed  and  updated  by  computer 
programmers  made  CoHOST  an  ideal  candidate  for  consideration  as  a  “black  box”  that  could  be 
used  to  stuff  data  into  and  receive  results  out  of. 

CoHOST  also  was  an  attractive  tool  because  of  the  potential  applicability  of  its  design 
constructs  for  use  in  other  work  domains.  The  work  group  modeled  by  the  original  project 
consists  of  23  members  of  a  U.S.  Army  heavy  maneuver  battalion’s  command  and  control 
elements  lead  by  the  battalion’s  commanding  officer.  As  battalion  commander,  this  individual 
not  only  commanded  the  entire  1,000  member  combat  maneuver  battalion,  but  also  acted  as  team 
leader  for  the  select  group  of  individuals  that  populated  the  battalion’s  tactical  operations  center 
(TOC)  during  combat  operations.  This  work  domain  is  described  as  a  time  pressured,  high 
stress,  decision  oriented  environment  where  information  is  received  from  the  outside  world  into 
the  team,  is  processed  by  it  resulting  in  decisions  that  are  made  primarily  by  the  battalion 
commander  and  then  the  results  of  those  decisions  are  passed  back  into  the  outside  world  in  the 
form  of  commands,  directives,  and  status  reports.  These  same  components  of  time  pressure, 
stress,  and  requirements  for  decision  making  and  communication  can  be  used  to  describe  many 
different  work  group  situations  that  exist  not  only  in  the  military,  but  also  in  other  governmental 
agencies  and  the  civilian  sector.  Hospital  emergency  room  teams  of  doctors,  nurses,  and  support 
staff  have  the  same  pressures  to  quickly  assimilate  information,  make  decisions,  and  then  act 
upon  those  decisions.  Other  examples  of  work  domains  that  could  be  addressed  by  a  CoHOST 
like  simulation  include: 

•  Nuclear  power  plant  control  room. 

•  Combat  ship  command  and  control  centers. 


•  Civil  Emergency  Action  Centers. 

•  Railroad  Dispatch  Centers. 

•  Air  Traffic  Control  Centers. 

i  For  the  work  domain  investigated  in  the  current  CoHOST  simulation,  some  of  the 

important  considerations  are  what  are  the  communication  requirements,  who  are  the  decision 

makers,  and  what  is  the  most  important  activity  to  be  performed  by  the  decision  makers.  A  list 

-f 

of  some  of  these  activities  include: 

•  Decision  making? 

•  Battlefield  assessment? 

•  Development  of  accmate  mental  models? 

•  Sharing  of  mental  models? 

•  Evaluating  impact  of  decisions? 

•  Inter  and  intra  team  communication? 

The  original  intent  and  use  of  CoHOST  centered  on  whether  the  human  operators  could  benefit 
and  perform  more  effectively  in  this  new  woik  domain  than  they  could  in  the  old,  existing 
battalion  TOC  work  domain.  The  methodologies  in  this  thesis  now  demonstrate  how  the 
simulation  can  be  expanded  from  its  original  intent  and  address  how  to  optimize  that  domain  to 
maximize  predicted  human  performance. 
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2.  Description  Of  The  Computer  Simulation. 


The  new  Command  and  Control  Vehicle  (C2V)  was  designed  to  have  a  two  man  crew  for 
vehicle  driving  and  movement  control  in  the  front  cab  with  up  to  four  computer  workstations  and 
two  auxiliary  “jump”  seats  in  the  rear  woiking  compartment.  An  auxiliary  40KW  power 
generator  provides  power  for  the  computer  systems  and  associated  radio  transmission  equipment 
for  operation  either  while  the  vehicle  is  moving  or  statioruuy  with  the  main  engine  powered 
down.  From  a  hardware  design  viewpoint  this  integration  of  a  modem  vehicle  platform  with 
state  of  the  art  communications  capabilities  directly  addresses  anticipated  current  and  future 
battlefield  command  and  control  requirements  well  into  the  next  century.  However,  this  vehicle 
and  its  systems  are  at  the  heart  of  a  change  of  the  operational  paradigm  in  command  and  control 
Tactical  Operations  Centers  (TOC’s)  that  the  Army  is  now  imdergoing.  Figure  1  illustrates  the 
components  in  an  Army  battalion  level  TOC  of  today. 
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Figure  1  -  Tactical  Operations  Center  (TOC)  Diagram 
(Middlebrooks  et  al.,  1999) 

The  rectangles  represent  the  different  vehicles  in  the  unit.  Personnel  are  listed  inside  the  vehicle 
box  and  the  communication  systems  for  each  vehicle  are  listed  beside  it.  This  select  group  of  24 
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people,  along  with  the  vehicles  and  communications  systems  they  use,  represent  diose  people 
directly  concerned  with  battlespace  management  and  it  is  this  working  group  that  is  modeled  in 
CoHOST. 

^  For  the  50  years  since  World  War  n  the  fundamental  nature,  organization,  and  mode  of 

operation  of  command  organizations  has  remained  unchanged.  Staffs  are  organized  on  a  basic 
four  section  structure  and  TOC’s  generally  only  operate  in  a  totally  static  mode  with  the  amoxmt 

*  of  time  required  to  move  them  to  keep  up  with  a  mobile  battlefield  going  up  almost 
exponentially  firom  lower  to  higher  command  levels.  However,  current  initiatives  are  changing 
all  that  and  while  new  vehicles  and  hardware  systems  address  the  ability  of  the  conunand 
structures  to  improve  their  operations,  these  initiatives  do  not  necessarily  provide  the 
environment  in  which  a  human  in  the  loop  operator  can  necessarily  function  in  a  more  effective 
manner. 

This  project  addressed  the  ability  of  the  human  component  of  the  new  operational 
systems  to  perform  under  a  new  operational  paradigm.  As  communications  systems  are  passing 
greater  and  more  accurate  volumes  of  information  in  real  time  the  question  to  be  asked  is  ‘can 
the  soldier  absorb  this  information  and  be  able  to  react  to  the  stream  of  data  being  presented  to 
him/her  also  in  real  time?’  Can  these  activities  be  performed  while  the  vehicle  is  moving  over 
extended  distances  and  during  extended  time  periods?  Do  the  combined  effects  of  fatigue,  noise, 
and  vibration  that  are  sustained  by  an  operator  cause  that  person  to  become  what  is  described  as 
a  “cognitive  causality”?  The  CoHOST  computer  models  and  project  looked  at  some  of  these 
issues  and  made  recommendations,  which  are  summarized  here,  that  predicted  the  performance 
potential  of  human  operators  in  this  new  working  environment. 

2.1.  Application  of  a  Taxonomy  of  Human  Performance. 

f 

A  With  work  first  published  in  1954,  Edwin  Fleishman  (Fleishman,  1975)  began  what 
would  turn  into  a  lifetime  effort  focused  on  the  development  of  taxonomic  descriptors  of  work 

*  performance.  The  resulting  taxonomy  (Fleishman  and  Quaintance,  1 984)  presents  a  set  of  skills 
and  abilities  that  can  be  used  to  describe  human  performance  characteristics  in  any  general  wodc 
situation.  Fleishman  stated  (Fleishman,  1975;  Fleishman,  1978)  that  some  kind  of  taxonomy  of 
human  performance  is  required  which  provides  an  integrative  framework  and  common  language 
applicable  to  a  variety  of  basic  and  applied  areas.  He  goes  on  further  to  state  that  it  does  appear 
that  predictions  and  generalizations  about  human  performance  may  be  enhanced  by  some  linkage 
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of  task  classification  systems  based  on  human  abilities  and  task  characteristics.  In  1988 
Fleishman  (Fleishman,  1988)  quoted  earlier  1947  work  by  others  with  the  observation  that 
apparatus  tests  of  perceptual  motor  abilities  had  been  found  to  have  considerable  validity  for 
predicting  the  success  of  pilots  and  bombardiers  in  getting  through  training  during  World  War  H. 

Comments  by  others  point  out  that  Fleishman’s  work  tends  to  be  neglected  in  the 
mainstream  of  human  information  processing  research,  perhaps  due  to  the  fact  that  the  skills  and 
abilities  in  the  taxonomy  are  only  based  on  factor  analyses  and  are  void  of  any  process 
description.  However,  the  tests  used  by  Fleishman  to  develop  the  taxonomy  belong  to  the  same 
type  of  performance  tests  that  are  studied  in  Wickens’  more  accepted  dual  task  experiments  and 
therefore  deserve  closer  scrutiny  (Sanders,  1997).  There  have  been  many  attempts  in  the  human 
factors  community  to  develop  similar  descriptions  of  human  performance  and  while  this 
taxonomy  may  not  be  generally  accepted  by  all  for  every  attempt  at  evaluations  of  human 
performance,  it  does  provide  a  set  of  skill  and  ability  descriptors  that  are  heavily  weighted  to 
cognitive  performance. 

Previous  work  at  the  U.S.  Army  Research  Laboratory  (ARL)  (Knapp,  1996a;  Knapp, 
1996b;  Knapp,  Johnson,  Barnette,  Wojciechowski,  Kilduff,  Bird,  and  Plott,  1997c;  Schipani  et 
al.,  1998),  and  the  U.S.  Army  Research  Institute  (ARI)  (Seven,  Akman,  Muckier,  Knapp,  and 
Bumstein,  1991)  identified  this  job  skill  and  ability  taxonomy  (Fleishman,  1984;  Fleishman  and 
Quaintance,  1984)  and  stated  that  it  showed  promise  to  provide  the  basis  for  workload  scaling  in 
Army  battalion  level  cotmnand  and  control  modeling  efforts.  This  taxonomy  consists  of  52 
skills  and  abilities  that  include  mental  processing,  sensory  perception  and  fine  and  gross  motor 
skills.  The  selection  of  this  taxonomy  was  influenced  by  its  detailed  decomposition  of  mental 
abilities  and  the  existence  of  behaviorally  anchored  rating  scales  (Knapp  et  al.,  1997c). 
Subsequently,  SO  of  the  52  skills  and  abilities  from  the  taxonomy  were  adopted  to  support  work 
that  was  performed  for  the  U.S.  Army  Intelligence  Center  at  Fort  Huachuca,  Arizona.  This  work 
sought  to  determine  basic  soldier  training  requirements  needed  to  provide  requisite  skills  and 
abilities  for  various  Military  Occupational  Specialty  (MOS)  at  the  Intelligence  Center’s  basic 
soldier  training  units.  As  shown  in  Figure  2,  the  taxonomy  was  grouped  into  eight  demand 
categories  of  reasoning,  speed-loaded,  conceptual,  communications,  visual,  auditory, 
psychomotor,  and  gross  motor.  Knapp  stated  that  (Knapp  et  al.,  1997c)  “  Each  skill  and  ability 
has  an  associated  behaviorally  anchored  rating  scale  that  ranges  fi»m  "1”  for  a  very  low  level 


16 


demand,  to  "7"  for  the  highest  demand.  Definitions  for  all  50  skills  and  abilities,  along  with  their 
behaviorally  anchored  scales,  is  documented  in  a  separate  review  of  this  taxonomy  (Seven  et  al., 
1991).” 
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Figure  2  -  Fleishman’s  Knowledge,  Skills,  and  Abilities  Taxonomy 
(Middlebrooks  et  al.,  1999) 

Using  Fleishman’s  taxonomy,  a  database  was  developed  using  questionnaires  using 
Likert  -  like  7  point  behaviorally  anchored  questions  and  was  administered  to  U.S.  Army  subject 
matter  experts  (SME).  This  questionnaire  associated  physical  and  mental  skills  and  abilities 
from  the  taxonomy  to  performance  tasks  such  as  ‘receive  and  record  a  radio  message’  that 
operators  would  be  expected  to  execute  in  the  performance  of  their  duties  in  a  TOC  during  the 
conduct  of  battlefield  operations.  This  database  then  provided  a  numerical  basis  for  a  computer 
simulation  model  to  calculate  a  workload  estimate  for  each  individual  based  on  the  tasks  being 
performed  at  the  instant  of  the  calculation.  The  time  interval  selected  for  workload  calculation 
updates  was  100  seconds.  Resulting  from  this,  over  the  course  of  a  simulation  run,  a  profile  of 
individual  woddoad  and  utilization  rates  was  established  for  each  member  of  the  workgroup  at  a 
100  second  resolution.  The  data  was  captured  so  that  the  workload  rates  could  be  decomposed 
into  the  individual  elements  of  the  taxonomy  so  that  the  amotmt  of  time  spent  by  the  individual 
in  the  different  cognitive  and  physical  performance  categories  could  be  determined.  These 
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woikload  and  utilization  profiles  were  then  analyzed  following  the  simulation  run  using 
multivariate  statistical  techniques  to  predict  whether  individuals  became  cognitively  saturated 
and  therefore  unable  to  effectively  perform  their  assigned  tasks. 

j#  2.2.  Tactical  Scenario. 

The  Battalion  task  force  mission  was  modeled  as  a  force-on-force  operation  occurring 
^  over  several  hours.  Different  scenarios  that  were  developed  include  the  phases  of  pre-operations 

planning,  movement-to-contact,  deliberate  defense,  and  hasty  attack.  Some  scenarios  reflect 
heavy  combat  actions  and  others  reflect  extended  movement  and  reconnaissance  type  operations 
as  shown  in  Figure  3.  A  model  input  file  consisting  of  scenario  voice  and  digital  messages 
expected  to  be  sent  to  and  from  the  battalion  during  the  course  of  the  tactical  mission  was 
generated  using  battalion-training  scenarios  for  Southwest  Asia  operations  and  OMS/MP 
(Operations  Mission  Summary  /  Mission  Profile)  movement  rates  as  provided  by  the  U.S.  Army 
Armor  Center  at  Fort  Knox,  Kentucky.  The  input  file  indicates  the  time  each  message  occurs, 
where  it  is  received  and  who  or  what  equipment  receives  it,  and  the  subsequent  routing  and  task 
flow  initiated  by  this  message.  Tasks  performed  in  response  to  these  messages  come  from  an 
external  source  (usually  a  radio,  digital  link,  or  coworicer),  and  are  labeled  "reactive”,  and  either 
"voice"  or  "digital".  In  addition  to  external  messages,  the  scenario  file  also  contains  "internal 
information  messages"  that  are  mental  "triggers"  for  personnel  to  periodically  perform 
"proactive"  (self-initiated)  tasks  that  are  an  essential  part  of  C2  operations  and  workstation 
database  manipulation  (Knapp,  Johnson,  Barnette,  Wojciechowski,  Kilduff,  and  Swoboda, 
1997a).  Examples  of  these  proactive  tasks  are  situation  assessment  checks,  updating 
documentation  (plans,  orders,  etc.),  preparing  status  reports,  and  calling  up  windows  of 
»  information  for  review. 
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Figure  3  -  Tactical  Scenario  Overlay 
(Middlebrooks  et  al.,  1999) 

2.3.  The  CoHOST  Computer  Model. 

Computer  modelers  used  the  discrete  event  programming  language  MicroSaint™  which 
provided  software  protocols  and  conventions  to  input  the  tasks,  task  sequences,  flow  logic,  and 
task  timing  and  workload  data  from  the  netwoik  diagrams  into  an  executable  model.  The 
computer  model  works  according  to  a  basic  “input-throughput-output”  scheme  as  shown  in 
Figure  4.  That  is,  the  inputs  to  the  model  are  message  events  from  the  scenario  input  file,  which 
present  an  information  event  stream  in  a  time  sequence  synchronized  to  mission  activity  phases. 
As  these  information  events  enter  the  model,  tasks  are  triggered  and  performed  in  a  pattern  that 
reflects  the  logic  for  task  branching,  interrupt  priorities,  time  outs,  and  collaborative  (interactive) 
tasks.  Any  information  event  that  triggers  a  staff  huddle  always  has  the  highest  priority  (Knapp 
et  al.,  1997a). 
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Figure  4  -  Conceptual  Model  of  TOC  Operations 
(Middlebrooks  et  al.,  1999) 

The  model  runs  on  an  IBM-compatible  PC  running  Windows  95  (or  higher).  During 
model  execution,  a  graphical  user  interface  (GUI)  screen  displays  the  progress  of  tasks  being 
performed  by  each  C2  section  and  individual  soldier  position,  as  information  messages  enter  the 
system.  Bar  and  pie  charts  on  the  GUI  display  allow  an  observer  to  get  an  initial  look  at  whether 
staff  sections  and  individuals  are  keeping  pace  or  falling  behind  in  their  information  processing, 
as  well  as  how  busy  or  idle  they  are  as  scenario  time  goes  on.  A  screen  print  from  this  real-time 
display  is  shown  in  Figure  5. 
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Figure  5  -  CoHOST  Model  Action  View  Display 
(Middlebrooks  et  al.,  1999) 


The  model  was  developed  in  three  steps  that  occurred  iteratively  and  in  parallel; 

(1)  Cognitive  task  analysis  and  workload  measurement  for  battalion 
command  and  control  tasks,  using  techniques  from  the  most  recent  human  performance  and 
related  literature; 

(2)  Obtaining  and  translating  scenarios  and  task  flow  data  from  pertinent 
documentation  and  battalion  command  and  control  subject  matter  experts; 

(3)  Exercising  the  MicroSaint^**  discrete  event  simulation  programming 
language  to  simulate  the  task  and  flow  data  from  steps  one  and  two.  Following  data  input,  the  C2 
computer  model  was  debugged  and  executed,  and  the  resulting  output  data  were  analyzed  using 
descriptive  and  comparative  statistics.  An  example  of  the  task  flow  logic  contained  in  one  of  the 
CoHOST  models  is  shown  in  Figure  6. 
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Figure  6  -  CoHOST  Model  Network  Flow  Diagram 


2.4.  Results  From  Original  CoHOST  Project. 

Each  CoHOST  model  was  executed  using  communication  messages  from  the  tactical 
scenario  as  driver  events  for  the  simulation.  The  dependent  measures  that  were  evaluated  were: 

1 )  Tasks  dropped  -  those  tasks  that  an  operator  did  not  complete  for  any  reason. 
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2)  Tasks  interrupted  -  those  tasks  whose  performance  was  interrupted  by  another 
task  or  event  of  higher  priority. 

3)  Number  of  task  queues  generated  -  the  number  of  times  an  incoming  task  was 
assigned  to  a  queue  wait  state  because  the  operator  identified  to  perform  the  task 
was  busy  performing  another  task  of  equal  or  higher  priority. 

4)  Task  backlog  work  -  off  time  -  the  amoirnt  of  time  it  took  for  an  operator  to 
eliminate  the  tasks  that  were  queued  up  for  execution. 


Additional  analyses  were  performed  to  assess  the  reasons  for  and  types  of  information  flow 
bottlenecks.  The  purpose  of  this  review  was  to  identify  why  tasks  got  dropped,  queued,  and/  or 
interrupted. 

Initially,  three  CoHOST  models  were  executed  with  varying  configurations  of 
organizational  configuration  and  implementation  of  digital  communications  equipment.  The 
results  are  summarized  in  Figure  7. 
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Figure  7  -  Percent  of  Time  Spent  In  Each  Performance  Category  of  the  Taxonomy 

(Knapp  et  al.,  1997c) 

Figure  7  presents  results  from  the  three  runs  with  information  organized  according  to  the 
taxonomy.  The  3  groups  of  bars  represent  information  from  officers,  NCO’s  and  junior  enlisted 
personnel.  The  3  bars  within  each  group  represent  the  data  from  each  run.  The  three  runs  were 
identified  as: 

1)  Baseline  Model  -  Personnel  and  equipment  configuration  according  to  the  then 
(1996)  mode  of  TOC  operation  with  analog  communications  equipment  (Knapp  et 
al.,  1997a). 
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2)  Traditional  Model  -  Same  as  the  baseline  model  but  with  first  generation  digital 
communications  equipment  (Knapp,  Johnson,  Barnette,  Woiciechowski,  Kilduff, 
and  Swoboda,  1997b). 

3)  hitegrated  Model  -  Reorganized  personnel  organizational  structure  to  capitalize 
on  enhanced  communications  equipment  capabilities  and  objective  digital 
communications  equipment  (Knapp  et  al.,  1997c). 

There  was  no  data  for  the  junior  enlisted  for  the  Integrated  model  run  because  all  the  junior 
enlisted  personnel  were  eliminated  by  the  personnel  reorganization  for  that  model  run. 

The  sections  of  each  bar  graph  are  color  coded  to  correspond  to  the  eight  categories  of  the 
taxonomy  as  indicated  in  the  legend.  The  top  category  represented  the  amount  of  time  spent 
performing  the  proactive  think  -  ahead  reasoning  cognitive  tasks  that  are  critically  important  for 
situation  analysis  and  decision  making  abilities.  The  next  category  is  the  cognitive  speed  loaded 
category  that  is  indicative  of  activity  that  requires  immediate  attention  for  quick  servicing  of  the 
activity  before  the  content  of  the  activity  becomes  obsolete.  Looking  at  the  three  bar  graphs  for 
the  3  model  runs  for  the  officers  at  the  left  side  of  Figure  7,  it  can  be  seen  that  in  the  baseline 
model  the  officers  were  modeled  as  being  able  to  spend  about  10%  of  their  time  performing  the 
proactive  diink  -  ahead  tasks  necessary  to  maintain  cognitive  awareness  of  the  battlefield  and 
develop  decisions  on  what  actions  to  take  next.  Subject  Matter  Expert  opinion  validated  that  this 
estimate  roughly  corresponded  to  the  circumstances  of  actual  battle.  The  middle  bar  from  the 
traditional  model  run  that  simulated  first  generation  digital  corrununications  equipment  for  the 
officers  show  that  this  activity  was  greatly  reduced  being  almost  totally  supplanted  by  the  speed 
loaded  activity  of  the  next  taxonomic  category.  The  third  run  fi:om  the  integrated  model  that 
simulated  the  full  capabilities  of  digital  communications  equipment  being  developed  and  a 
reorganization  of  personnel  to  take  full  advantage  of  it  shows  an  even  worse  situation  with 
almost  all  reasoning  activity  disappeared. 

The  explanation  for  this  phenomenon  comes  fi’om  a  realization  that  while  each  element  of 
the  communications  equipment  was  performing  exactly  as  it  was  designed,  the  design  was  based 
on  maximizing  hardware  system  performance  that  did  not  include  the  human  as  an  integral 
component  of  the  system.  The  result  was  that  increased  message  arrival  rates  coming  firom 
enhanced  communications  systems  were  forcing  the  decision  makers  to  focus  their  attention  to 
just  trying  to  keep  up  with  and  react  to  the  messages  with  the  result  that  there  was  no  time  left  to 
analyze  and  interpret  the  information  they  were  receiving.  Thus,  in  this  series  of  simulation  runs 
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the  officer  decision  makers  went  from  a  pseudo  proactive  think  -  ahead  reasoning  mode  to  an 
almost  total  reactive  speed  loaded  mode  while  trying  to  keep  up  with  the  increased  message 
traffic.  A  contributing  element  to  this  situation  was  the  elimination  of  the  junior  enlisted 
personnel  whose  primary  duties  were  to  function  as  equipment  operators.  With  the  limited  space 
and  seating  in  the  C2V  the  officer  decision  makers  were  required  to  sit  at  and  operate  their  own 
communications  consoles  and  had  to  personally  interact  with  the  incoming  message  traffic. 

2.5.  Summary  of  Original  CoHOST  Project. 

By  looking  at  which  individuals  were  predicted  to  be  workload  saturated  for  each  model 
run  condition,  a  project  conclusion  was  reached  that  increasing  automation  does  not  necessarily 
improve  human  decision  making  performance  and  may,  in  fact,  degrade  it. 
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3.  Method. 


An  experimental  design  is  developed  that  is  an  extension  of  a  basic  2^  Va  replicate 
fractional  factorial  design  that  includes  an  additional  level  to  allow  an  examination  of  the 
4  quadratic  components  of  the  main  effects.  This  design  is  further  optimized  by  unbalancing  the 

treatment  orders  to  allow  the  computer  simulation  requirements  to  stay  within  acceptable  limits 
and  still  provide  enough  data  to  support  the  desired  analysis. 

3.1.  Experimental  Design. 

The  purpose  of  this  effort  is  to  conduct  an  experiment  to  evaluate  work  environment 
conditions  to  achieve  optimum  human  performance.  The  work  environment  is  a  battalion  level 
command  and  control  center  for  the  U.S.  Army  during  combat  operations  and  is  modeled  by  the 
CoHOST  battle  simulation.  The  CoHOST  simulation  models  17  different  individuals  in  the 
command  center.  This  analysis  focuses  on  one  of  these  operators,  the  Battalion  Commander. 

The  independent  variables  will  be  five  types  of  tasks  that  personnel  perform  in  this  situation. 

The  dependent  variables  will  be  the  resulting  workload  and  total  utilization  as  determined  from 
the  performance  of  the  specified  job  tasks  by  the  simulated  human  operators  during  the  computer 
run.  Additional  dependent  variables  will  assess  the  number  of  tasks  that  become  queued, 
dropped,  or  interrupted  during  the  course  of  the  simulation. 

Although  early  limitations  inherent  in  using  computer  simulations  to  support 
experimental  designs  (Smith  and  Mauro,  1981)  have  largely  been  overcome  due  to  technical 
advances  in  hardware  and  software,  the  temptation  to  use  the  computer  to  investigate 
experimental  design  configurations  that  would  be  impossible  with  controlled  experimentation 
using  human  test  subjects  can  lead  to  complex  and  large  experimental  efforts.  As  a  result,  many 
of  Smith  and  Mauro’s  ideas  that  stemmed  from  the  batch  processing  orientation  processing 
capabilities  of  the  1980*s  still  have  meaning  in  the  high  speed  PC  and  Workstation  technological 
base  of  today.  Their  thoughts  on  ways  to  reduce  the  number  of  factors  and  required  number  of 
computer  runs  still  have  meaning  where  desired  input  conditions  and  outcomes  must  be  balanced 
against  the  ability  to  produce  those  outcomes  in  a  timely  manner.  The  simulation  support  plan  as 
presented  in  the  following  paragraph  will  be  iterated  throughout  this  paper  to  arrive  at  an 
optimum  mix  of  computer  support  requirements  versus  desired  experimental  outcomes. 


26 


As  a  first  step,  the  base  required  level  of  computer  simulation  support  needs  to  be 
identified.  A  CoHOST  simulation  run  of  one  replication  takes  approximately  10  minutes  to 
process  a  24  hour  battlefield  scenario  on  the  available  hardware  (see  paragraph  3.2.1).  Previous 
test  runs  of  the  simulation  have  determined  that  each  data  gathering  run  needs  to  include  15 
replications  of  the  simulation  to  provide  a  95%  probability  of  achieving  satisfactory  error  levels 
of  the  dependent  variables  due  to  random  number  generation  in  the  model.  Further,  in  order  to 
simulate,  for  example,  the  effects  of  3  subjects  for  each  treatment  condition  the  experiment  must 
be  executed  three  times  at  each  treatment  level  with  the  random  number  seed  set  to  a  new  value 
at  the  beginning  of  each  run.  Thus,  the  anticipated  required  number  of  computer  simulation  run 
replications  to  satisfy  this  requirement  is  32  x  15  x  3  =  1440.  At  10  minutes  per  run  replication 
this  requires  1440  x  10  =  14,400  minutes  or  240  hours  of  computer  time.  In  order  to  bring  this 
required  processing  time  down  to  a  manageable  level  a  %  replicate  fractional  factorial  of  the  2^ 
within  subjects  design  (n=3)  will  be  used.  With  this  design  60  hours  of  computer  time  is  still 
required  to  produce  360  replications  with  the  available  hardware  and  software. 

3.1.1.  Independent  Variables. 

The  experimental  design  for  this  study  includes  five  independent  variables  (IV).  These 
IV’s  are  listed  in  Table  1  and  comprise  the  five  major  categories  of  performance  evaluation  that 
the  CoHOST  model  uses  to  evaluate  task  load  performance.  In  the  CoHOST  model  these 
performance  categories  are  decomposed  into  32  task  performance  categories  to  provide  the  level 
of  detail  that  the  developers  of  the  simulation  were  seeking.  These  32  performance  categories 
and  their  mapping  to  the  higher  level  categories  is  shown  in  Figure  8.  When  this  experimental 
design  was  being  developed  the  choice  was  to  evaluate  all  32  tasks  as  independent  variables  or  to 
use  their  aggregated  values  in  the  five  top  level  categories.  The  original  model  developers  set  up 
these  top  level  categories  to  gather  knowledge,  skill  and  ability  data  from  subject  matter  experts 
according  to  the  human  performance  taxonomy  used  by  the  simulation  (Fleishman  and 
Quaintance,  1984).  As  the  evaluation  of  a  2^  full  factorial  design  was  deemed  impractical  due  to 
limitations  on  computer  resources  and  time  constraints  forcing  a  more  efficient  Va  replicate 
design,  an  attempt  to  evaluate  a  2^^  design  was  beyond  the  capabilities  of  this  effort.  Although 
the  Va  replicate  fractional  factorial  2^  design  allows  a  resolution  in  experiment  that  is  able  to 
resolve  all  the  main  effects,  trying  to  bring  a  2^^  design  into  a  manageable  scope  with  a  1/8  or 
1/16  fitictional  factorial  design  would  not  allow  a  full  resolution  m  evaluation.  For  these 
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reasons,  an  experimental  design  utilizing  the  5  top  level  factors  was  chosen  even  though  it 
required  manual  translation  of  data  into  and  out  of  the  32  task  categories  used  by  the  simulation. 


Communicate  and  Report 

01 -Receive  and  Record/Anaiog 

Communicate  and  Report 

b2-Pass  information 

Communicate  and  Report 

03-Listen-Receive  information 

Communicate  and  Report 

04-Secondary  Monitor 

Communicate  and  Report 

05-Log  Message 

Communicate  and  Report 

06-Route  (Outside  the  Section) 

Communicate  and  Report 

07-Send  Message 

Communicate  and  Report 

08-Verbai  Order 

Communicate  and  Report 

09-Roll  Up  Reports 

Communicate  and  Report 

10-Call  to  Conference 

Communicate  and  Report 

26-Receive  Digital  Message 

Communicate  and  Report 

27-Input  data  Into  Computer 

Communicate  and  Report 

28-Send  Digital  Information 

Decide  and  Recommend  /  Direct 

1 1 -Decide  Action 

Decide  and  Recommend  /  Direct 

12-Decide 

Decide  and  Recommend  /  Direct 

13-Recommend  Action 

Evaluate  and  Estimate  Impact 

14-Estimate  Impact 

Evaluate  and  Estimate  impact 

15-Data  Gathering/  Analog 

Evaluate  and  Estimate  impact 

16-Find  Options 

Evaluate  and  Estimate  impact 

17-Compare  Alternatives 

Evaluate  and  Estimate  impact 

18-Discuss 

Evaluate  and  Estimate  impact 

29-Data  Gathering/  Digital 

identify/understand  Situational  Picture 

19-Read/Analog 

Identiiy/Understand  Situational  Picture 

20-Scan 

Identify/Understand  Situational  Picture 

21-Update/  Analog 

identify/Understand  Situational  Picture 

22-Check  Status 

Identify/Understand  Situational  Picture 

23-Problem  Definition 

Identify/Understand  Situational  Picture 

24-Listen/Monitor  -  Analog 

identify/Understand  Situational  Picture 

30-Read/  Digital 

Identify/Understand  Situational  Picture 

31-Scan  Digital 

Identify/Understand  Situational  Picture 

32-Monltor  Digital 

Manage  Resources 

25-Manage  Resources 

Figure  8  -  CoHOST  Task  Performance  Categories 

The  first  IV  is  Conununicating  and  Reporting  Tasks  (CAR).  This  variable  assesses  the 
ability  of  the  operator  to  express  himself  or  herself  through  both  auditoiy  and  automated  means 
of  communication.  The  next  variable  is  Deciding  and  Recommending  or  Directing  Tasks 
(DRD).  The  DRD  variable  measures  the  performance  load  on  the  operator  from  making 
decisions  and  in  passing  the  results  of  those  decisions  to  superiors  in  the  form  of 
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reconunendations  or  to  subordinates  in  the  form  of  directives  or  orders.  The  variable  Evaluating 
and  Estimating  Impact  (EEI)  measures  the  primarily  cognitive  activities  associated  with  the 
assessment  of  what  the  results  of  actions  taken  and  directives  issued  by  the  operator  will  be. 

The  task  Identify  and  Understand  Situational  Picture  (USP)  describes  those  tasks  and 
activities  associated  with  the  assessment  of  the  activities  going  on  around  the  operator  and  the 
primarily  cognitive  activities  of  trying  to  understand  the  relevance  of  these  activities  and  in  the 
«  development  and  maintenance  of  a  mental  model  that  reflects  all  the  activities  that  are  going  on 

around  the  operator  that  is  of  significance. 

The  Manage  Resources  (MRS)  variable  requires  an  extended  description.  When  the 
CoHOST  simulation  was  being  developed  this  was  a  variable  element  that  was  designed  to  track 
the  operator’s  ability  to  manage  the  available  physical  resources  and  how  the  presence  or 
absence  of  these  resources  enabled  the  operator  to  perform  his  or  her  duties.  However,  this 
capability  was  never  implemented  in  the  simulation  model.  Although  the  data  elements  and 
variable  tables  were  coded  into  the  simulation,  no  logic  was  ever  implemented  to  use  these 
variables.  When  the  experimental  design  for  this  study  was  being  developed  it  was  decided  to 
use  this  variable  in  the  design  to  determine  baseline  values  to  establish  a  threshold  of 
significance  for  the  lack  of  fit  (LOF)  calculation.  By  using  the  MRS  variable  to  establish  the 
significance  level  for  LOF  for  interpreting  results  from  regression  runs  with  the  SAS  statistical 
package,  actual  values  for  significance  can  be  mathematically  determined  that  take  into  account 
the  random  variability  ‘noise’  in  the  CoHOST  model.  This  process  consists  of  recording  the  ‘p’ 
value  from  the  SAS  run  for  the  MRS  variable  and  then  selecting  the  next  highest  value  that  is 
available  in  standard  statistical  tables  (Winer,  Brown,  and  Michels,  1991)  which  is  then  used  in  a 
manually  evaluated  decision  rule  evaluation  of  the  significance  of  the  LOF  parameter.  After  its 
^  use  for  determining  this  threshold  of  significance,  the  MRS  variable  is  then  dropped  from  all 

further  analyses  in  this  study. 

The  treatment  levels  for  the  experimental  design  are  set  up  by  adjusting  up  and  down  the 
original  values  in  the  CoHOST  model  that  are  used  to  determine  the  values  for  each  of  the  FV’s 
as  shown  in  Table  1.  The  amount  of  adjustment  up  and  down  for  the  baseline  values  was 
arbitrarily  determined  to  be  20%  which  was  judged  to  provide  two  data  points  approximately 
equally  distanced  on  a  100%  scale.  As  only  the  battalion  commander  operator  is  being  evaluated 


29 


in  this  study,  the  performance  values  for  these  five  categories  as  they  apply  to  this  operator  are 
used  to  set  up  these  treatments. 

Table  1  -  Independent  Variables  and  Their  Treatment  Levels 


Indenendent  Variables 

Treatment  Level 

Communicate  and  Report  Tasks  (CAR) 

Decide  and  Recommend  /  Direct  Tasks  (DRD) 

Evaluate  and  Estimate  Impact  Tasks  (EEI) 
Identify/Understand  Situational  Picture  Tasks  (USP) 
Manage  Resources  Tasks  (MRS) 

(Baseline  -  20%,  Baseline  +20%) 
(Baseline  -  20%,  Baseline  +20%) 
(Baseline  -  20%,  Baseline  +20%) 
(Baseline  -  20%,  Baseline  +20%) 
(Baseline  -  20%,  Baseline  +20%) 

3.1.2.  Description  of  Vi  Replicate  Fractional  Factorial  Design. 

A  full  factorial  2*  design  would  provide  32  treatment  combinations,  therefore  this  Vi 
replicate  design  provides  8  treatment  combinations.  Figure  9  shows  the  SAS  ADX  Experiment 
Design  module  specifying  a  2  level,  5  factor  design  that  generates  8  treatment  combinations. 

This  indicates  that  a  Resolution  3  design  is  possible  at  the  Vi  fractional  factorial  level  with  only 
the  main  effects  being  estimable.  However,  this  thesis  is  only  interested  in  evaluating  the  main 
effects  due  to  the  nature  of  the  factors  involved.  All  of  the  higher  interactions,  including  the  two 
way  interactions,  have  no  meaning  for  this  analysis.  For  example,  an  attempt  to  interpret  the 
interaction  between  “Communicating  and  Reporting”  and  “Evaluating  and  Estimating  Impact” 
would  presuppose  individual  actions  that  are  not  quantifiable  in  the  T(X:  based  work  situation. 
For  this  reason  only  the  main  factor  effects  are  of  interest.  This  further  justifies  a  Resolution 
level  3  experiment  which  can  be  conducted  at  the  Va  fractional  factorial  level. 

% 


* 
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Figure  9  -  Configuration  Options  for  a  5  Factor  Experimental  Design 


The  treatment  combinations  for  this  design  are  shown  in  Table  2.  See  Appendix  A  for 
the  derivation  of  these  combinations. 

Table  2  -  Vi  Replicate  Treatment  Combinations 


Comm& 

Report 

Decide  & 
Recommend  / 
Direct 

Evaluate  & 
Estimate 
Impact 

Identify/ 

Understand 

Situational 

Picture 

Manage 

Resources 

Treatment 

Factor: 

A 

B 

C 

D 

E 

Combinations 

CAR 

DRD 

EEI 

USP 

MRS 

1 

1 

-1 

-1 

-1 

-1 

2 

-1 

1 

1 

-1 

-1 

3 

1 

1 

-1 

1 

-1 

4 

-1 

-1 

1 

1 

-1 

5 

-1 

1 

-1 

-1 

1 

6 

1 

-1 

1 

-1 

1 

7 

-1 

-1 

-1 

1 

1 

8 

1 

1 

1 

1 

1 

3.1.3.  54  Replicate  Identity  Relationships. 


Using  the  experimental  design  capabilities  in  SAS  the  confoimding  rules  were  identified 


and  are  shown  in  Figure  10. 

m 
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Equation  1  -  Identity  Relationships:  2^  - 14  Replicate  Fractional  Factorial  Design 

Applying  these  identity  relationships  to  all  of  the  main  effects  and  interactions,  the  ANOVA 
Summary  Table  for  this  design  is  shown  in  Table  3. 

Table  3  -  ANOVA  Summary  Table  for  14  Replicate  Fractional  Factorial  Design 


Cl:  Xl+X2  +  X4  = 

AxBxD 

C2:  Xl+X3  +  X5  = 

AxCxE 

C1-K:2:  X2  +  X3  +  X4  +  X5  = 

BxCxDxE 

I  =  AxBxD,  AxCxE,  BxCxDxE 

n  = 

5 

Treatments  = 

8 

Source 

df 

A 

(BxD,  CxE,  AxBxCxDxE) 

1 

B 

(AxD,  AxBxCxE,  CxDxE) 

1 

C 

(AxBxCxD,  AxE,  BxDxE) 

1 

D 

(AxB,  AxCxDxE,  BxCxE) 

1 

E 

(AxBxDxE,  AxC,  BxCxD) 

1 

BxC 

(AxCxD,  AxBxE,  DxE) 

1 

BxE 

(AxDxE,  AxBxC,  CxD) 

1 

Subj  /  Treatments  = 

Treatments  x  (n-1) 

32 

Total: 

39 

Verification: 

Treatments  x  n  -1 

39 

3.1.4.  Augmenting  The  V4  Replicate  Fractional  Factorial  Design. 

The  experimental  design  as  shown  in  Table  3  provides  an  economical  means  of 
determining  the  linear  components  of  the  main  effects  of  the  independent  variables.  However,  it 
does  not  provide  any  indication  of  whether  or  not  the  simulation  model  is  generating  any 
significant  higher  order  components  of  these  effects  because  of  the  fact  that  the  2^  - 14  replicate 
fractional  factorial  design  is  only  capable  of  generating  a  linear  response  surface.  Traditionally, 
to  evaluate  if  higher  order  effects  exist  in  the  model  would  require  an  expansion  of  the  model  to 
3  or  more  levels  per  treatment  condition  along  with  a  full  factorial  evaluation  of  the  results. 
Because  of  limitations  already  discussed  this  is  not  possible  using  traditional  factorial  designs. 
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There  is  the  possibility,  however,  to  investigate  possible  quadratic  effects  in  the  model  which 
could  provide  the  justification  for  sequential  experimentation  into  a  higher  order  model.  This 
possibility  is  to  augment  the  existing  number  of  treatment  conditions  with  one  additional 
condition  by  using  the  center  point  of  the  data  as  shown  in  Table  4. 

Table  4  -  Augmentation  of  V4  Replicate  Treatment  Combinations 


4 


Comm& 

Report 

Decide  & 
Recommend  / 
Direct 

Evaluate  & 
Estimate 
Impact 

Identify/ 

Understand 

Situational 

Picture 

Manage 

Resources 

Treatment 

Factor; 

A 

B 

C 

D 

E 

Combinations 

CAR 

DRD 

EEI 

USP 

MRS 

1 

1 

-1 

-1 

-1 

-1 

2 

-1 

1 

1 

-1 

-1 

3 

1 

1 

-1 

1 

-1 

4 

-1 

-1 

1 

1 

-1 

5 

-1 

1 

-1 

-1 

1 

6 

1 

-1 

1 

-1 

1 

7 

-1 

-1 

-1 

1 

1 

8 

1 

1 

1 

1 

1 

9 

0 

0 

0 

0 

0 

Since  the  treatment  levels  for  this  model  design  were  determined  by  adding  and 
subtracting  20%  to  the  original  observed  data  for  the  model  as  described  in  Paragraph  2.2.2,  the 
center  points  for  these  +1/-1  treatment  levels  exist  in  the  original  data.  By  designating  these  data 
as  treatment  level  ‘0’  a  ninth  treatment  condition  can  be  added  to  the  fractional  factorial  design 
as  an  augmented  treatment  condition  with  all  the  independent  variables  at  a  treatment  level  of 
‘O’.  This  will  provide  the  means  to  investigate  whether  any  quadratic  components  exist  for  the 
main  effects  in  the  empirical  model. 

3.1.5.  Dependent  Variables. 

In  the  CoHOST  model  these  five  parameters  are  not  just  individual  numbers  but  consist 
of  products  of  the  amoimt  of  time  required  to  perform  tasks  for  each  category  and  the  KSA 
values  describing  the  physical  and  mental  level  of  effort  required  to  perform  the  tasks.  The 
dependent  variables  (DV)  for  the  experimental  design  correspond  to  those  major  categories  of 
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data  programmed  into  the  CoHOST  model.  The  first  dependent  variable  is  Taskload.  Originally 
termed  workload  by  the  model  developers,  this  term  refers  to  the  amount  of  ph3^ical  and  mental 
performance  imposed  on  the  operator  by  the  tasks  being  performed.  This  experimental  design 
includes  two  measures  of  taskload. 

The  seven  dependent  variables  are  summarized  in  Table  5.  The  first  taskload  measure  is 
Taskload  from  Knowledge,  Skills,  and  Abilities  (TLKSA).  The  treatment  levels  for  this  DV  are 
«  established  by  applying  the  20%  adjustment  factor  to  the  KSA  values  from  the  performance 

taxonomy  in  the  CoHOST  database.  As  the  KSA  values  are  only  used  in  the  taskload  calculation 
and  none  of  the  other  output  parameters,  the  model  runs  conducted  with  these  adjustments 
provide  an  operator  response  to  the  effects  to  more  or  less  mental  and  physical  effort  expended 
on  the  performance  of  the  tasks. 

Table  5  -  Dependent  Variables 


« 


Dependent  Variables 
Taskload/ KSA  (TLKSA) 


Taskload/ TT(TLTT) 


Utilization  (UTDL) 


Description _ 

Expressed  as  a  dimensionless  quantity  that  is 
an  expression  representing  the  performance 
of  individiial  tasks  with  specified  skill  levels 
overtime. 

Expressed  as  a  dimensionless  quantity  that  is 
an  expression  representing  the  performance 
of  individual  tasks  when  the  amount  of  time 
to  perform  that  task  is  varied. 

Percent  of  time  the  operator  was  busy  over 
the  simulation  run. 


Tasks  Queued  (NOQUE)  Number  of  tasks  that  were  queued  on  the 

operator 

Tasks  Dropped  (TSKDRP)  Number  of  tasks  that  the  operator  dropped 

during  the  simulation. 


Tasks  Suspended  (TSKSUP)  Number  of  tasks  that  were  suspended  during 

a  run  as  a  result  of  a  higher  priority 
interrupt. 

Tasks  Interrupted  (TSKINT)  Number  of  times  the  operator  was 

interrupted  while  performing  a  task. 
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The  second  taskload  measure  is  Taskload  from  Task  Time  (TLTT).  The  treatment  levels 
for  this  DV  are  established  by  applying  the  20%  adjustment  factor  to  the  amount  of  time,  in 
seconds,  required  to  perform  each  task  in  the  CoHOST  database.  The  model  runs  conducted 
with  these  adjustments  provide  an  operator  response  to  the  effects  of  more  or  less  time  required 
to  perform  each  task.  As  the  task  time  parameter  is  also  used  in  all  the  other  CoHOST  output 
categories,  the  computer  runs  made  with  these  treatment  combinations  also  provided  data  for  the 
«  remaining  DV’s.  These  remaining  DV’s  are  Utilization  (UTIL),  Number  ofTasks  Queued  on 

The  Operator  during  the  simulation  run  (NOQUE),  the  Number  ofTasks  Dropped  (TSKDRP)  by 
the  operator  during  the  run  (i.e.,  not  completed  for  any  reason),  the  Number  ofTasks  Suspended 
(TSKSUP)  during  the  run,  and  the  Number  ofTasks  that  were  Interrupted  (TSKINT)  during  the 
run.  While  the  CoHOST  model  simulates  the  activities  of  23  operators  in  the  command  and 
control  center,  each  of  these  dependent  measures  describe  activities  for  the  single  operator  under 
investigation  in  this  thesis  which  is  the  battalion  commander. 

3.2.  Experimental  Procedures. 

The  first  step  in  collecting  data  with  this  or  any  other  computer  model  is  to  determine 
how  many  replications  of  the  simulation  must  be  made  in  each  computer  run  in  order  to  accoimt 
for  random  variability  in  the  simxilation.  The  simulation  is  then  executed  according  to  the 
experimental  design  using  the  number  of  required  replications  and  data  is  collected. 

3.2.1.  Determination  Of  Simulation  Replication  Count. 

With  stochastic  computer  simulations,  multiple  runs  must  be  conducted  for  each 
combination  of  the  independent  variables  in  order  to  account  for  the  variability  induced  by  the 
random  number  generation  (Whicker  and  Sigelman,  1991a).  When  analysis  is  conducted  on  data 
generated  by  a  simulation  due  to  random  effects  then  the  variance  of  the  output  data  must  be 
controlled  so  that  it  falls  within  a  desired  precision  limit  (Banks  et  al.,  1996).  Stated  succinctly, 

^  a  computer  simulation  model  involving  Monte  Carlo  determinations  needs  to  be  repeated  or 

‘replicated’  as  many  times  as  necessary  to  get  the  required  precision  (Kelton,  1995).  This  can  be 
achieved  by  making  multiple  replications  of  the  simulation  runs  by  holding  the  independent 
variable  levels  constant  and  changing  the  random  niunber  seed  at  the  start  of  each  replication 
run.  When  a  sufficient  number  of  replications  have  been  executed  then  the  mean  of  the  output 
data  from  the  replications  can  be  expected  to  fall  within  the  desired  confidence  limit.  For  this 
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study  it  is  desired  to  have  the  output  data  from  the  simulation  exhibit  a  95%  probability  of  falling 
within  the  confidence  limit  which  gives  a  specified  error  level,  e,  equal  to  ±.05  of  the  mean. 
Following  Banks’  procedures,  (pages  429-449),  the  required  niimber  of  replications  can  be 
^  determined  that  needs  to  be  conducted  to  support  the  intended  analysis. 

An  initial  simulation  run  of  5  replications  was  made  with  a  starting  random  number  seed 
of  1 .  The  model  automatically  used  the  next  random  niunber  at  the  end  of  each  replication  as  the 
«  starting  seed  for  the  next  replication.  The  resulting  data  for  workload,  utilization,  tasks  queued, 

tasks  dropped,  and  tasks  interrupted  for  the  Battalion  Commander  is  shown  in  Table  6.  A 
replication  analysis  was  performed  for  each  of  these  dependent  variables  to  determine  the 
number  of  replications  required  to  satisfy  each  of  these  measures. 

Table  6  -  Data  From  Initial  5  Replication  Simulation  Run 


Battalion  Commander 

Workload 

(•Util) 

W!mW 

Tasks 

Dropped 

(•Opdata) 

Replication  1 

3561179.34 

92.30% 

50 

90 

11 

11 

Replication  2 

3549435.31 

92.07% 

48 

90 

11 

13 

Replication  3 

3556732.44 

92.39% 

54 

90 

10 

11 

Replication  4 

3548795.69 

93.21% 

52 

88 

11 

9 

Replication  5 

3526501.95 

91.95% 

51 

91 

11 

13 

Mean 

3548528.95 

92.38% 

51.00 

89.80 

10.80 

11.40 

Standard  Deviation 

13354.88 

0.0049 

2.24 

1.10 

0.45 

1.67 

5%  Error  Limit  = 

177426.45 

0.0462 

2.55 

4.49 

0.54 

0.57 

.05,  relative  to  the  mean 

*  (filename)  = 

name  of  model  output  data  file 

The  desire  is  to  determine  the  number  of  replications  required  so  that  the  relative  error 
(relative  to  the  mean)  for  any  of  the  dependent  variables  does  not  exceed  5  percent.  The  iterative 
formula  to  determine  the  number  of  replications  is  (Banks  et  al.,  1996)  (eq.  12.29,  p.  449)  is 
shown  as  Equation  2. 
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Equation  2  -  Initial  Estimate  for  Number  of  Required  Replications,  R 


where, 

R 

number  of  replications  required  to  achieve  the  desired  error  level 

Z 

Z  statistic 

So 

Standard  Deviation  of  the  computed  parameter  across  the 
simulation  replications 

6 

= 

Error  Level  Threshold 

O 

Percent  Error  Level  of  the  mean  value  of  the  computed  parameter 
across  the  simulation  replications. 

This  expression  is  iteratively  computed  with  the  value  of  the  computed  replications  being 
substituted  for  R  until  the  value  for  R  satisfies  the  greater  than  or  equal  to  condition  at  which 
time  the  value  for  R  becomes  the  required  nxunber  of  replications  necessary  to  compensate  for 
the  random  effects  of  the  simulation. 

Example  Calculation  For  Replication  Analysis  For  Workload  Parameter: 

From  Table  6: 

So  =1354.88 

6=117426.45 

a  =  .05;  0/2  =  .025;  l-a/2  =  .975 

.•.Z.975  =  1-96,  from  Z  table,  page  966  (Winer  et  al.,  1991) 

Thus, 

R^{(Zo^2xSo)/e}2 

R  >  {(1.96  X  13354.88)  /  177426.45}^=  .02176  «  1 
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So,  use  R  =  1 .  Since  this  is  less  than  SO,  use  the  t  distribution,  plug  back  into  the 
formula  and  evaluate.  From  the  t  table,  (Winer  et  al.,  1991),  page  967,  tan,\  - 
12.71 

R  >  {(r97.5,2  X  So)  /  e}^ 

R  >  {(12.71  X  13354.88)  /  177426.45}  ^  =  .91524  »  1 
5  >  1  relationship  is  verified.  As  1  run  is  required  and  5  have  been 
made,  no  additional  runs  are  required  to  satisfy  this  parameter. 

Table  7  shows  the  replication  analysis  for  all  the  dependent  variables.  From  this  analysis  it  is 
determined  that  the  parameter  “Number  of  Queues”  is  the  defining  variable  and  will  require  15 
replication  runs  to  satisfy  the  criteria. 

Table  7  -  Replication  Analysis  For  Initial  5  Replication  Run 


1  R&[(Z„,2xSo)/e]' 

Battalion 

Commandar 

IQI 

^0/2.df 

So 

e 

R 

R  Adjusted 

Conclusion 

Workload: 

mmnn 

mi 

mmin 

Since  R  <  50,  use  t  disbribution  and  recalculate 

/.5  ^  1  relationship  is  satisfied.  No  more  runs  required 

^  97.5,1 

mm 

e 

KB 

R  Adjusted 

1.96 

13354.88 

177426.45 

0.02176 

1 

12.71 

13354.88 

177426.45 

0.91524 

1 

Utilization: 

mm 

mmm 

mmnnm 

Since  R  <  50,  use  t  disbribution  and  recalculate 

/.5  ^  2  relationship  is  satisfied.  No  more  runs  required 

BH 

^97.5,1 

mm 

e 

R 

R  Adjusted 

0.0049 

0.0462 

0.04395 

1 

12.71 

0.0049 

0.0462 

1.84795 

2 

Number  of  Queues 

HHB 

Since  R  <  50,  use  t  disbribution  and  recalculate 
.*.5  ^  15  ->  No;  Set  R=15  and  reevaluate 

So 

8 

R 

R  Adjusted 

1.96 

^97.5,2 

2.24 

2.55 

2.95394 

3 

■Si 

2.24 

2.55 

14.21761 

15 

^97.5,14 

mm 

8 

R 

A 15  ^4  ->  Yes,  therefore  Use  R  =  15 

2.14 

2.24 

2.55 

3.52141 

4 

Tasks  Interrupted 

Since  R  <  50,  use  t  disbribution  and  recalculate 

A  5  2: 10  ->  No;  Set  R=10  and  reevaluate 

So 

8 

R 

R  Adjusted 

1.96 

^  97.5,1 

1.10 

4.49 

0.22867 

1 

12.71 

1.10 

4.49 

9.61567 

10 

^97.5,9 

K9 

8 

R 

A 10  2: 1  ->  Yes,  therefore  Use  R  -  10 

2.26 

1.10 

4.49 

0.30402 

1 

Tasks  Suspended 

HH 

mmn 

■■■ 

HHH 

mmniii 

Since  R  <  50.  use  t  disbribution  and  recalculate 
.*.5  St  13  ->  No;  Set  R=13  and  reevaluate 

BIB 

mm 

8 

KB 

R  Adjusted 

1.96 

^  97.5,2 

0.45 

0.54 

2.63484 

3 

mm 

0.45 

0.54 

12.68176 

13 

^97.5,12 

KHI 

8 

R 

A 13  ^  4  ->  Yes,  therefore  Use  R  «  13 

2.18 

0.45 

0.54 

3.25953 

4 

3.2.2.  Simulation  Execution. 


Because  MicroSaint™  simulations  have  the  input  data  embedded  in  the  model  code,  a 
separate  simulation  model  was  configured  for  each  simulation  run.  Each  simulation  run  of  15 
*  replications  was  conducted  with  the  copy  of  the  model  that  was  configured  for  the  treatment 

condition  associated  with  the  run.  A  total  of  9  models  was  configured  for  the  9  treatment 
conditions  for  each  of  two  treatment  manipulations.  The  dependent  variable  “Taskload  from 

4 

Knowledge,  Skills  and  Abilities  (TLKSA)”  was  developed  by  adjusting  the  behaviorally 
anchored  taxon  values  for  each  performance  task  from  Fleishman’s  taxonomy  (Fleishman  and 
Quaintance,  1984).  These  adjustments  consisted  of  adding  and  subtracting  20%  to  the  base 
values  already  existing  in  the  simulation  in  order  to  set  up  two  treatment  conditions  according  to 
the  experimental  design.  An  excerpt  from  the  spreadsheet  used  to  set  up  these  parameters  is  in 
Table  8.  The  Duty  Name  indicates  parameters  for  the  first  independent  variable,  Communicate 
and  Report  (CAR).  The  Scale  Number  column  indicates  taxon  #1  from  the  50  elements  of 
Fleishman’s  taxonomy.  The  Scale  Name  is  the  associated  name  for  the  Scale  Number.  The 
Detail  Duty  represents  those  task  duties  that  the  Battalion  Commander  performs  in  the  model 
while  conducting  the  oral  comprehension  component  of  communicating  and  reporting.  The 
Detail  Score  column  is  the  original  KSA  performance  score  according  to  the  7  point  behaviorally 
anchored  scale  for  that  performance  task.  The  Treatment  Factor  is  the  adjustment  factor  for  the 
experimental  design  here  showing  an  increase  of  20%  for  the  +1  treatment  level.  Finally,  the 
Detail  Score- Adj  column  shows  the  adjusted  task  score  as  a  multiple  of  the  Detail  Score  and  the 
Treatment  Factor.  A  modified  spreadsheet  was  then  configured  with  the  Detail  Score- Adj  value 
replacing  the  Detail  Score  column  that  was  then  processed  with  an  Excel  macro  to  generate  the 
^  MicroSaint™  compatible  code  statements  which  were  then  inserted  into  the  simulation  model  in 

the  WL_CMD  (JASS  Workload  Data)  function  table. 


« 
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Table  8  -  Exceipt  of  Input  Data  For  Configuration  of  KSA  Treatment  Levels 


1  Communicate  and  Report 

ORAL  COMPREHENSION 

01 -Receive  and  Record/Analog 

Kingi 

1.20 

rm 

0010 

ORAL  COMPREHENSION 

02-Pass  information 

0.00 

1.20 

ORAL  COMPREHENSION 

03-Listen-Receive  Information 

1.84 

1.20 

Communicate  and  Report 

0010 

ORAL  COMPREHENSION 

04-S6condary  Monitor 

tm 

1.20 

Communicate  and  Report 

0010 

ORAL  COMPREHENSION 

05-Log  Message 

wm 

1.20 

wmM»GKV:in!r.iami 

0010 

ORAL  COMPREHENSION 

06-Route  (Outside  the  Section) 

0.00 

1.20 

mill 

[iSaSMlKBI 

0010 

ORAL  COMPREHENSION 

07-Send  Message 

0.00 

1.20 

I*F1 

Communicate  and  Report 

0010 

ORAL  COMPREHENSION 

08-Verbai  Order 

lEa 

1.20 

mwm 

Communicate  and  Report 

0010 

ORAL  COMPREHENSION 

09-Roll  Up  Reports 

0.00 

1.20 

EESM 

Communicate  and  Report 

ORAL  COMPREHENSION 

10-Cail  to  Conference 

1.84 

1.20 

2.21 

ORAL  COMPREHENSION 

26-Receive  Digital  Message 

0.00 

1.20 

warn 

mssmmkmM 

ORAL  COMPREHENSION 

27-Input  data  Into  Computer 

0.00 

1.20 

MiTil 

[Communicate  and  Report 

ORAL  COMPREHENSION 

28-Send  Digital  Information 

1.20 

The  corresponding  MicroSaint™  code  after  processing  by  the  macro  for  this  example  is 
displayed  in  Table  9.  All  of  the  numbers  past  the  4*  significant  digit  are  the  result  of  roundoff 
error  from  Excel  that  is  passed  into  the  data  table  by  the  macro  and  are  insignificant  in  the 
processing  performed  by  MicroSainr”.  Each  full  table  for  this  manipulation  consists  of  1600 
elements  comprising  the  50  taxons  (ScaleName)  times  the  5  independent  variables  (DutyName) 
times  the  subset  number  of  DetailDuty  elements  that  pertains  to  each  condition  from  the  task 
duty  set  of  32  tasks.  In  the  example  above  there  are  13  detail  duties  making  up  the  oral 
comprehension  component  of  communicating  and  reporting  for  the  battalion  commander.  See 
appendix  B  for  full  listing  of  the  KSA  setup  data  using  treatment  condition  1  as  an  example. 

Table  9  -  Excerpt  of  KSA  Treatment  Data  After  Excel  Macro  Processing 


WL[1.2.11;=0: 

WL[1 ,3.1];=2.20800004005432; 

WL[1.6,1]:=0: 

WL[1,7.1]:=0: 

WL[1 ,10,1]:=2.20800004005432: 

WL[1,26,1]:=0: 

WL[1.27,1]:=0: 

WL[1.28.11:=0: 
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The  second  manipulation  for  dependent  variables  Taskload  from  Task  Time  (TLTT), 
Utilization  (UTIL),  Number  of  Queues  Generated  During  The  Run  (NOQUE),  Number  of  Times 
A  Task  Was  Intermpted  (TSKINT),  Number  of  Times  A  Task  Was  Suspended  (TSKSUP),  and 
Number  of  Times  A  Task  Was  Dropped  (TSKDRP)  was  performed  by  adjusting  the  task 
performance  time  parameters  up  and  down  according  to  the  experimental  design  using  the  same 
+/-  20%  values  as  before.  An  excerpt  from  the  spreadsheet  that  was  used  in  this  step  is  shown  in 
Table  10.  The  Number  column  is  the  task  identification  number  used  by  the  model.  The  Name 
column  contains  a  text  name  corresponding  to  the  task  number.  The  Operatorl  column  reflects 
the  primary  operator  performing  this  task.  The  data  here  indicates  operator  29,  the  battalion 
commander.  The  task  time  data  for  all  of  the  other  operators  was  left  imchanged  as  only  the 
battalion  commander’s  task  time  performance  data  was  being  adjusted  up  and  down  in  the 
experimental  design.  The  Operator2  column  contains  a  placeholder  for  a  secondary  operator  for 
the  performance  of  this  task.  For  all  of  the  battalion  commander  tasks  there  was  no  secondary 
operator.  The  Function  column  shows  the  ftmction  category  code  for  the  task.  The  WtFac 
column  indicates  a  priority  weighting  factor  for  the  task.  The  Mean  column  is  the  time,  in 
seconds,  for  the  performance  of  the  task.  The  Interrupt  Type  column  shows  the  priority  level  for 
this  task  if  it  tries  to  interrupt  another  task.  The  Original  Mean  column  shows  the  original  mean 
task  time,  in  seconds,  before  the  experimental  design  adjustment.  The  IV  Category  coliunn 
shows  the  independent  variable  this  task  feeds  into,  here  showing  independent  variable  #1, 
Communicate  and  Report  (CAR). 

Table  10  -  Example  of  Configuration  of  Task  Time  Treatment  Levels 


Receive  and  Record 

29 

1 

A 

2 

1.2 

0.50 

Receive  and  Record 

29 

1 

A 

2 

1.2 

0.50 

\mmm 

Pass  Info 

29 

2 

A 

■  ■KCTB 

2 

1.2 

1  863 

Pass  Info 

29 

2 

A 

IBEHEHIB 

2 

masim 

1.2 

■■■EEID 

mmm 

Listen  Receive 

29 

3 

A 

2 

1.2 

The  corresponding  MicroSaint™  code  after  processing  by  the  macro  for  this  example  is 
displayed  in  Table  1 1 .  See  Appendix  C  for  a  full  listing  of  the  Task  Time  setup  data  for 
treatment  condition  1 . 
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Table  11  -  Example  of  Task  Time  Treatment  Data  After  Excel  Macro  Processing 


m\  nniiiiipi  HiHiii— 

TaskData[802,Wtlndx]:»A; 

T  askData[802,Tm  Indx];s0.504; 

T  askData[802,lntlndxl:=2; 

Ta8kData[825,Oplndx1:s29; 

TaskData[825,DJFIndx]:=1 ; 

T  askDataI825,T  m  lndx]:=0.504: 

T  askData[825,lntlndx];>2; 

TaskData[81 7,Oplndx]:=29; 

1  1  ^  j  I  d  1 

TaskData[81 7,Wtlndx]:=A; 

TaskData[81 7.Tmlndx]:=2.01 6; 

iiTyrriririiiiiiTiiiiii—i 

TaskData[863,Oplndx]:B29; 

TaskData[863,DJFIndx];s2; 

TaskData[863.Wtlndx]:-A; 

T  askData[863,T m  1  ndx]: » 1 9.656; 

T  a8kData[824,Oplndx];s29; 

T  a8kDatai824,DJFIndx];=3; 

hiriTTifTir  ViiirriiiniBMi 

Ta8kData[824,Tmlndx];=2.016; 

T  a8kData[824,lntlndx1:s2; 

The  Treatment  Multiplier  colunm  contains  a  multiplier  that  is  used  to  adjust  the  mean  task  time 
up  or  down,  here  showing  a  value  of  1 .2  indicating  an  adjustment  of  +20%.  The  Treatment 
Product  column  shows  the  adjusted  task  time,  in  seconds,  which  is  the  product  of  the  original 
mean  times  the  treatment  multiplier.  This  value  has  also  been  stored  back  into  the  Mean  column 
by  the  spreadsheet  for  processing  by  an  Excel  macro  which  translates  this  information  into  coded 
values  which  were  then  inserted  into  the  MicroSaint™  model’s  TASKDATACMD  function 
table. 

The  9*  treatment  combination  is  termed  the  zero  or  center  point  (CP)  value  and  consists 
of  the  original  task  time  and  KSA  data  in  the  model.  As  this  data  was  used  to  generate  both  the 
KSA  based  and  Task  Time  based  treatment  levels  for  the  independent  variables  it  was  applicable 
to  be  used  for  the  KSA  variable  and  all  of  the  Task  Time  dependent  variables.  There  were  a  total 
of  18  models  configured  for  this  data  collection  with  9  being  for  the  KSA  data  treatments  and  9 
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for  the  Task  Time  based  data  treatments.  The  models  were  run  and  data  collected  and  stored  for 
subsequent  analysis. 

3.3.  Apparatus. 

The  CoHOST  simulation  model  is  written  in  the  MicroSaint™  programming  language  by 
Micro  Analysis  and  Design  Corporation.  MicroSaint™  is  a  discrete  event  simulation  language 
that  is  designed  to  support  human  performance  studies  and  is  applicable  for  a  wide  range  of 
human  performance  domains  (Laughery  and  Corker,  1992).  The  version  used  to  support  this 
thesis  was  Release  3.1  Build  A  with  ActionView  and  OptQuest,  Standard  Version,  that  was 
released  on  October  27, 1999.  The  computer  system  environment  supporting  the  MicroSaint™ 
simulation  is  an  IBM  compatible  personal  computer  using  the  Microsoft™  Windows  2000 
Professional  Edition  operating  system.  The  hardware  specifications  of  the  computer  system  used 
is: 

•  CPU:  Pentium  n  running  at  450  MHZ. 

•  Bus  Rate:  133  MHZ. 

•  Memory:  390  MB. 

•  Disk  storage:  64  GB. 

Using  this  equipment  the  amount  of  computer  time  estimated  to  run  all  the  treatment 
combinations  is  47.5  hours  as  shown  in  Table  12. 

Table  12  -  Simulation  Rim  Analysis 


-  8  Runs  Of  15  Replications  Repeated  1  Time  For  Factoriai  Treatments. 

120  Replications. 

-  1  Run  Of  15  Replications  Repeated  10  Times  For  Center  Point. 

165  Replications. 

•  Total  Required  Replications  = 

285  Replications. 

1-  Computer  Time  Required  @10  Minutes  Per  Replication : 

>  Factorial  Treatments: 

8  X 15  X 1  X 10  s  1200  minutes  =  20.0  Hours 

>  Center  Point  Treatment: 

1  X 15  X 11  X 10  *  1650  minutes  *  27.5  Hours 

>  Total 

»  47.5  Hours 
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4.  Results. 


The  output  data  will  be  analyzed  to  provide  information  to  generate  a  set  of  polynomial 
regression  equations  that  will  describe  the  performance  of  the  model  under  its  entire  range  of 
performance.  These  equations  will  be  algebraically  manipulated  to  predict  the  maximum 
combinations  of  task  times  and  knowledge,  skills,  and  abilities  that  will  produce  the  minimum 
workload  for  each  operator  for  this  work  environment  within  the  constraints  of  the  simulation. 

The  data  produced  from  the  simulation  runs  was  evaluated  with  a  series  of  statistical 
reviews  using  the  SAS  statistical  software  package  (version  8.01  on  IBM  compatible  personal 
computer  running  Windows  2000).  The  resulting  regression  equations  for  each  dependent 
variable  was  then  evaluated  by  taking  the  partial  derivative  of  each  equation  and  setting  it  equal 
to  zero  to  produce  a  system  of  simultaneous  equations  describing  the  independent  variables  for 
each  dependent  variable.  Solving  these  equations  yields  the  optimum  value  of  the  independent 
variables  to  minimize  the  dependent  variable. 

These  analyses  begin  with  tests  for  multicollinearity  in  the  data  followed  by  development 
of  regression  equations  for  each  dependent  variable.  Each  independent  variable  was  tested  for 
significance  against  each  dependent  variable  using  the  general  linear  model  procedure.  Finally, 
the  regression  lack  of  fit  was  tested  for  significance  to  determine  if  there  were  qriadratic 
components  existing  in  the  response  for  the  independent  variable  main  effects.  A  response 
surface  regression  analysis  was  then  performed  to  evaluate  these  quadratic  components. 

4. 1 .  Data  Collection. 

Each  model  was  run  for  a  sufficient  number  of  replications  to  ensure  that  IS  valid 
replications  was  completed  for  each  treatment  condition.  Even  though  this  simulation  was 
subjected  to  a  thorough  testing  sequence  during  its  development  it  was  impossible  to  test  all  of 
the  possible  task  interactions  that  might  be  generated  when  parameters  such  as  task  time  are 
changed  that  would  result  in  different  task  time  performance  and  differing  completion  times  and 
follow  on  sequences  of  resulting  tasks.  With  32  different  detail  level  tasks  that  are  coded  into  the 
simulation  it  is  theoretically  possible  that  32  factorial  different  task  interaction  combinations  can 
exist  in  the  simulation.  As  a  result,  when  these  data  collection  runs  were  conducted  there  was  no 
way  to  know  in  advance  if  all  the  task  interaction  combinations  resulting  from  unique  random 
number  seeds  and  task  time  adjustments  that  would  result  in  the  run  had  been  previously  tested 
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and  would  not  generate  an  error.  As  a  result  there  were,  in  fact,  replications  in  almost  every  run 
that  generated  errors  that  MicroSaint™  identified  as  warnings  requiring  operator  intervention  to 
resume  the  simulation  run  and  continue  on  to  the  next  replication.  Most  of  the  runs  did  generate 
3  or  4  bad  replications  and,  as  a  result,  each  run  was  set  to  execute  for  20  replications  to  ensure 
that  15  good  replications  were  produced.  There  were  a  total  of  27  simulation  runs  producing  15 
replications  each  that  included  8  runs  for  the  fractional  factorial  treatments  for  KSA  data,  8  runs 
for  the  fhictional  factorial  treatments  for  the  Task  Time  data,  and  1 1  runs  for  the  center  point 
data.  While  most  runs  did  produce  3  or  4  bad  replications,  one  run  executed  error  free  and  one 
run  required  almost  50  replications  to  obtain  the  required  15  good  replications. 

Each  treatment  condition  model  was  configured  to  produce  the  full  range  of  output 
capable  from  the  simulation  even  though  only  a  small  portion  of  the  data  from  operator  29,  the 
battalion  commander,  was  required  for  this  study.  Given  the  level  of  effort  required  to  configure 
and  run  the  models  the  conclusion  was  reached  to  gather  and  store  all  possible  data  on  all  the  23 
operators  for  possible  future  research  and  data  mining  efforts.  As  a  result  of  this  decision 
approximately  130  megabytes  of  data  was  generated  and  stored  from  each  simulation  run.  Two 
of  the  16  output  files  from  each  run,  UT1L.RES  and  OPDATA.RES  (the  MicroSaint™  .RES 
extension  means  a  results  file)  contained  all  of  the  data  required  to  support  this  study.  This  data 
was  first  transcribed  into  a  data  run  spreadsheet.  The  data  run  spreadsheet  for  the  KSA  treatment 
1  simulation  run  is  shown  in  Table  13.  This  spreadsheet  shows  replications  6, 7, 8,  and  17  as 
having  failed  and  replication  20  being  not  necessary  to  achieve  the  required  1 5  good  replications. 
The  15  data  elements  were  then  averaged  at  the  bottom  of  the  spreadsheet  to  provide  the 
resulting  data  parameters  for  that  run.  The  data  run  spreadsheets  for  all  the  27  simulation  runs  is 
at  appendix  D. 
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Table  13  -  KSA  Treatment  1  Data 


Random#  Seed  =  1.0 
T1-KSA-PMMMM 


Battalion 

Commander 

B 

1 

Taskload 

(Workload) 

(Util) 

Replication 

1 

1 

31.99 

Replication 

B 

B 

31.88 

B 

B 

31.95 

Replication 

4 

4 

31.88 

Replication 

B 

B 

31.68 

Replication 

B 

B 

Replication 

B 

B 

Replication 

B 

B 

Replication 

B 

B 

31.92 

Replication 

Kl 

B 

31.99 

Replication 

ED 

B 

31.92 

Replication 

m 

B 

31.96 

Replication 

EEI 

EEl 

31.77 

Replication 

EQ 

ED 

31.81 

Replication 

Ea 

m 

31.84 

Replication 

ES 

EO 

31.80 

Replication 

EQ 

B 

Replication 

18 

14 

31.88 

Replication 

EEl 

32.00 

Rt| 

B 

lAveraqe 

wa 

H 

31.88 

Following  completion  of  all  the  simulation  runs  the  data  elements  from  each  run  wer 
consolidated  into  a  master  spreadsheet  that  provided  the  data  matrix  for  the  data  analysis 
computer  runs.  This  master  spreadsheet  is  shown  in  Table  14.  As  an  example  the  result  from  the 
KSA  Treatment  1  Data  example,  31 .88,  is  reflected  in  the  treatment  1  row  for  Taskload  (KSA). 
The  treatment  condition  column  in  this  table  indicates  the  treatment  levels  for  the  5  dependent 
variables  according  to  the  experimental  design  using  a  shorthand  notation  where  “M”  indicates  a 
minus  20%  treatment  level,  “P”  indicates  a  plus  20%  treatment  level,  and  “Z”  indicates  a  zero  or 
center  point  treatment  level. 
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Table  14  -  Data  Matrix  For  All  Simulation  Runs 


Battalion 

Cdr 

Run 

# 

Treat¬ 

ment 

Cond. 

Treat  - 
ment 
Range 

Ran. 

# 

Seed 

Taskload 

[Workload  - 
Task  Time] 
(Uti!) 

Taskload 

[Workload  - 
KSA] 
(Util) _ 

utilization 

(Util) 

Number  of 
Queues 
(Opdata) 

Tasks 

Interrupted 

(Opdata) 

Tasks 

Suspended 

(Opdata) 

Tasks 

Dropped 

(Opdata) 

1 

1 

0.9250 

49.33 

90.33 

10.47 

9.60 

[Treatment 

2 

n 

40.084 

0.9256 

47.13 

■EHEBBI 

10.33 

9.40 

3 

■ddy.idTJKicai 

an 

38.588 

0.9213 

55.13 

88.47 

10.47 

14.20 

4 

m 

36.793 

0.9242 

51.73 

88.47 

10.67 

12.20 

5 

1 

39.810 

0.9242 

42.80 

90.87 

10.40 

8.93 

6 

m 

39.439 

0.9244 

50.60 

90.27 

10.80 

10.93 

7 

1 

39.217 

0.9240 

48.27 

89.07 

10.40 

11.07 

8 

KtVM 

1 

38.891 

0.9183 

58.47 

87.27 

9.47 

15.33 

9 

1 

39.439 

39.439 

0.9244 

50.60 

90.27 

10.80 

10.93 

iimSiHill 

wssm 

2 

39.391 

39.391 

0.9240 

49.87 

90.47 

10.60 

11.13 

mm 

mm 

39.372 

39.372 

0.9242 

49.87 

89.87 

10.80 

11.13 

WKm 

4 

39.351 

39.351 

0.9243 

49.53 

90.13 

10.93 

10.53 

umismi 

mm 

■TOM 

39.407 

39.407 

0.9246 

50.93 

89.60 

10.73 

10.47 

mm 

a>>>>a 

VJiVM 

wm 

39.367 

39.367 

0.9244 

49.67 

89.60 

10.80 

10.93 

wimrsm 

mam 

K»VM 

7 

39.320 

39.320 

0.9246 

49.07 

90.27 

10.73 

10.93 

16 

8 

39.346 

39.346 

0.9239 

49,73 

89.87 

10.87 

10.60 

17 

KiEB 

MM 

40.207 

40.207 

0.9235 

50.00 

69.87 

10.80 

10.87 

18 

mm 

39.340 

39.340 

0.9238 

50.07 

89.67 

10.73 

11.20 

[Treatmentl  19 

ZZ2ZZ 

■n 

39.362 

39.382 

0.9246 

50.40 

89.80 

10.60 

10.80 

*  Data  In  Shaded  Ce] 


Is  From  Computer  Runs  20-27-  Fractional  Factorial  KSA  Runs 


In  order  to  be  able  to  interpret  the  results  of  the  responses  of  the  dependent  variables  as 
they  relate  to  each  other,  the  data  for  each  dependent  variable  was  standardized  so  that  it  can  all 
be  examined  along  the  same  period  of  performance.  Using  a  mean  of  ‘0’  and  a  standard 
deviation  of  ‘  1’  for  translation  of  the  data  into  unit  normal  form,  a  new  dataset  was  created.  The 
dependent  variable  data  matrix  after  standardization  is  shown  in  Table  IS.  This  dataset  will  be 
used  for  all  of  the  analytical  reviews. 


Table  15  -  Data  Matrix  After  Standardization 


car 

isa 

mrs 

subj 

titt 

util 

tskint 

tsksup 

tskdrp 

1 

El 

El 

El 

El 

1 

0.01558 

-1.61817 

0.72212 

0.79778 

-0.40003 

-1.01866 

-1 

1 

1 

-1 

-1 

1 

1.17781 

-1.45600 

1.10148 

-0.99729 

-0.66798 

-0.83082 

-1.15316 

1 

1 

-1 

1 

-1 

1 

-0.97139 

1.52767 

-1.61728 

1.62852 

-1.36595 

-0.40003 

2.07483 

-1 

-1 

~T^ 

1 

-1 

1 

-3.55015 

1.68720 

0.21630 

0.51255 

-1.36595 

0.21540 

0.72984 

-1 

1 

-1 

-1 

1 

1 

0.78418 

-1.68973 

0.21630 

-2.41850 

1.42595 

-0.61543 

-1.46923 

1 

-1 

1 

-1 

1 

1 

0.25118 

-1.38444 

0.34276 

0.14166 

0.72798 

0.61543 

-0.12424 

-1 

-1 

-1 

1 

1 

1 

-0.06775 

1.45348 

0.08985 

-0.62311 

-0.66798 

-0.61543 

-0.03009 

1 

1 

1 

1 

1 

1 

-0.53609 

1.07270 

-3.51409 

2.72479 

-2.76191 

-3.47715 

2.83476 

0 

0 

0 

0 

0 

1 

0.25118 

0.03518 

0.34276 

0.14166 

0.72798 

0.61543 

-0.12424 

0 

0 

0 

0 

0 

2 

0.18223 

0.02467 

0.08985 

-0.09795 

0.96064 

0.00000 

0.01026 

0 

0 

0 

0 

0 

3 

0.15493 

0.02051 

0.21630 

-0.09795 

0.26266 

0.61543 

0.01026 

0 

0 

0 

0 

0 

4 

0.12476 

0.01592 

0.27953 

-0.20955 

0.56512 

1.01545 

-0.39323 

0 

0 

0 

0 

0 

5 

0.20521 

0.02817 

0.46921 

0.24997 

-0.05143 

0.40003 

-0.43358 

0 

0 

0 

0 

0 

6 

0.14775 

0.01942 

0.34276 

-0.16359 

-0.05143 

0.61543 

-0.12424 

0 

0 

0 

0 

0 

7 

0.08022 

0.00913 

0.46921 

-0.36053 

0.72798 

0.40003 

-0.12424 

0 

0 

0 

0 

0  I 

8 

0.11758 

0.01482 

0.02662 

-0.14390 

0.26266 

0.83082 

-0.34616 

0 

0 

0 

0 

0  i 

9 

1.35452 

0.20325 

-0.22629 

-0.05528 

0.26266 

0.61543 

-0.16459 

0 

0 

0 

0 

0 

10 

0.10896 

0.01351  j 

-0.03661 

-0.03230 

0.03000 

0.40003 

0.05734 

0 

0 

0 

0 

0 

11 

0.16930 

0.02270 

0.46921 

0.07601 

0.18123 

0.00000 

-0.21166 

4.2.  Development  of  Polynomial  Regression  Equations. 

A  set  of  polynomial  regression  equations  was  developed  for  each  dependent  variable  that 
will  be  used  to  perform  a  response  surface  analysis  for  that  variable.  The  sections  below  review 
the  requirements  for  developing  these  equations  and  describe  the  general  form  that  these 
equations  are  expected  to  take. 

4.2. 1 .  Data  Collection  Requirements  For  Polynomial  Regression  Equations. 

First,  there  must  be  one  more  data  point  than  the  number  of  parameters  fitted 

(Kleinbaum,  Kupper,  Muller,  and  Nizam,  1998)  (p.  290).  This  data  collection  procedure 
t 

involves  five  predictors,  Xi  -  X5,  corresponding  to  the  five  independent  variables  of  the  design. 
Thus,  the  number  of  Beta  Weight  parameters  to  be  fitted  in  this  equation  is  equal  to  the  five  Beta 
«  coefficients  for  the  predictors,  Bi  -  B5,  plus  the  Beta  Weight  for  the  intercept.  Bo,  which  gives 

six  parameters  to  be  fitted.  For  the  Va  replicate  fi:actional  factorial  design  there  are  eight 
treatment  conditions  generating  eight  data  points  which  satisfies  this  criterion. 

Second,  there  must  be  one  more  level  in  the  treatment  conditions  than  the  highest  order  of 
the  polynomial  (Kleinbaum  et  al.,  1998)  (p.  290).  The  polynomial  to  be  generated  here  is  linear 
giving  a  first  order  polynomial.  Therefore,  there  must  be  at  least  two  treatment  levels  for  each 
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predictor.  As  this  is  a  2^  factorial  design  all  the  factors  have  two  treatment  levels.  Therefore, 
this  requirement  is  satisfied. 

Third,  there  must  be  some  replication  of  the  treatment  conditions  (i.e.,  number  of  subjects 
per  cell)  in  order  to  provide  a  reliable  estimation  for  the  error  (Kleinbaum  et  al.,  1998)  (p.  290). 
As  previously  stated  there  are  eleven  replications  of  the  center  point  treatment  thereby  satisfying 
this  requirement. 

4.2.2.  Determination  of  The  Regression  Equations. 

For  this  5  factor  within  subjects  design,  the  general  form  of  the  polynomial  regression  / 
multiple  linear  regression  equation  is  shown  as  Equation  3. 

Y  =  Bo  +  BiXi  +  B2X2  +  B3X3  +  B4X4  +  B5X5 
Equation  3  -  General  Form  of  Expected  Polynomial  Regression  Equation 

This  equation  is  repeated  for  each  dependent  variable. 

4.2.3.  Maximizing  The  Regression  Equation  Variables. 

The  first  step  in  determining  the  maximiun  value  for  each  variable  in  the  model  is  to  take 
the  partial  derivative  of  each  variable: 

Y’xi  =  d/(X,)  =  Bi  +  B2X2  +  B3X3  +  B4X4  +  B5X5 
Y’x2  =  d/(X2)  =  BiXi  +  B2  +  B3X3  +  B4X4  +  B5X5 
Y’x3  =  d/(X3)  =  BiX,  +  B2X2  +  B3  +  B4X4  +  B5X5 
Y’x4  =  d/(X4)  =  BiXi  +  B2X2  +  B3X3  +  B4  +  B5X5 
Y’xs  =  d/(X5)  =  BiXi  +  B2X2  +  B3X3  +  B4X4  +  Bs 
Set  each  partial  derivative  equal  to  zero  to  maximize  the  function: 

Bj  +  B2X2  +  B3X3  +  B4X4  +  BsXs  =  0 
BjXi  +  B2  +  B3X3  +  B4X4  +  B5X5  =  0 
B]Xi  +  B2X2  +  B3  +  B4X4  +  B5X5  =  0 
B)Xi  +  B2X2  +  B3X3  +  B4  +  B5X5  =  0 
BiXi  +  B2X2  +  B3X3  +  B4X4  +  B5  =  0 

This  now  gives  5  equations  with  5  unknowns.  The  procedure  is  to  now  solve  for  the  5  unknowns 
and  substitute  the  Beta  Weights  from  regression  analysis  and  evaluate  the  value  for  each  variable 
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Xi  to  X5.  Thus,  the  equation  for  each  dependent  variable  will  generate  a  set  of  optim;im  values, 
as  predicted  by  the  computer  simulation,  for  the  independent  variables  that  will  be  needed  to 
minimize  the  dependent  variable. 

a  4.2.4.  Significance  Test  OfThe  Regression  Model. 

The  data  fi'om  the  simulation  runs  of  the  augmented  Va  rq>licate  fiiactional  factorial  design 
are  included  in  a  miiltiple  regression  analysis  (Pedhazur,  1997)  to  construct  a  first  order 

4 

polynomial  function  that  describes  the  behavior  of  the  simulation.  For  the  resulting  model  two 

descriptors  of  model  validity  are  computed.  The  coefficient  of  correlation,  is  a  measure  of 

the  percentage  of  the  variation  in  the  actual  response  function  that  is  explained  by  the  regression 

model.  The  F  ratio  is  an  indication  of  the  statistical  significance  of  the  constructed  model 

(Brown  and  Nachlas,  1985).  The  F  test  will  select  between  the  two  hypotheses: 

Ho:  Regression  model  is  not  statistically  different  from  zero  (Null  Hypothesis). 

Hi  :  Regression  model  is  significant  (Alternate  Hypothesis). 

Decision  Rule:  If  the  calculated  value  of  the  F  ratio  exceeds  the  corresponding  tabulated 
value  for  the  F  distribution,  then  Hi  is  believed  true.  Otherwise,  Ho  is 
accepted  as  true. 

The  regression  model  ANOVA  summary  table  is  shown  in  Table  16  that  was  used  to  generate 
the  F  ratios  to  support  the  hypothesis  tests. 


« 


« 
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Table  16  -  Regression  Model  Sununaiy  Table 


dumber  of  Repititions  per  Cell  (n)  = 

3 

#  of  Treatment  Combinations  That 
Have  Repeated  Observations  (T)  = 

9 

Total  Dat 

a  Points  (Treatments)  (N  =  n  x  T)  = 

27 

Source 

df 

E 

Regression 

(5) 

B, 

1 

MSbi/MSe, 

B2 

1 

IIIBBBBB! 

Bj 

1 

wggBBSIM 

B4 

1 

MSb4/MSe„ 

B5 

1 

MSb5/MSe„ 

Residual 

(21) 

Lack  Of  Fit  (LOF) 

3 

Error  (Replications)  =  (n-1)  x  T  = 

18 

Total: 

(26) 

Verification:  N-1  = 

26 

4.2.S.  Alternative  Regression  Analysis. 

In  order  to  refine  the  model  and  reduce  the  number  of  required  simulation  runs,  a 
refmement  of  the  regression  model  is  utilized.  This  alternative  model  uses  an  unbalanced 
number  of  treatment  combinations  where  each  of  the  fractional  factorial  combinations  is  repeated 
only  once  and  the  center  point  treatment  is  repeated  multiple  times.  This  design,  illustrated  in 
Table  17,  termed  a  2^'^  plus  Center  Point  With  Repeated  Observations  design,  has  the  advantages 
of  optimizing  the  simulation  execution  with  a  fewer  number  of  runs  as  shown  in  Table  7.  The 
real  advantage  is  that  it  provides  the  ability  to  look  for  quadratic  effects  as  a  result  of  having  the 
third  treatment  level  that  is  provided  by  the  center  point  treatment.  The  disadvantage  of  this 
design  is  that  it  is  slightly  less  powerful  with  only  10  degrees  of  freedom  in  the  error  term  versus 
18  in  the  original  design. 

The  analysis  procedure  to  test  this  model  is  summarized  as  follows: 

1 .  Test  die  linear  model  for  significance  of  the  main  effects. 

2.  Test  the  LOF  for  significance. 
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a.  If  the  LOF  is  significant  then  a  higher  order  effect  is  predicted. 

1)  Test  the  2"**,  3"*,  etc.  effects  for  significance. 

2)  Perform  ‘Data  Snooping’  to  identify  what  the  higher  order  polynomial 

might  look  like. 

However,  focus  on  effects  only  up  to  the  quadratics  of  the  main  effects  as  human  performance  is 
difficult  to  quantify  beyond  second  order  interaction  and  effects. 

Table  17  -  Alternative  Regression  Model  Summary  Table 


Number  of  Repititions  per  Cell  (n): 

1/4  Replicate  Treatments  =  n(l/4)  = 

8 

1/4  Replicate  Treatment  Combinations  = 

1 

*  Pick  Center  Point  Treatments  *  n(cp)  = 

11 

Number  of  Data  Points  That  Have  Repeated  Observations 
(T)  (i.e.,  the  replication  of  the  Center  Point)  - 

1 

Treatment  Combinations  (N  =  (n(l/4)  +  n(cp))  x  T)  = 

19 

Source 

df 

F 

Regression 

(5) 

Bi 

1 

MSbi/MSe„ 

Bi 

1 

MS02  !  MSgrr 

Bj 

1 

MSb3  /  MS^rr 

B4 

1 

MSb4/MSe„ 

Bj 

1 

MSb5/MSe„ 

Residual 

(13) 

Lack  Of  Fit  (LOF) 

3 

MSlof  /  MSsfr 

**((n(cp)-l)xT  = 

10 

Total: 

(18) 

Verification:  N-1  = 

18 

*  Pick  n(cp)  to  cause  FI 

llatio  for  Beta  Weights  to  be  below  S.  A  value  of  1 1 

causes  the  error  term  to  be  equal  to  10  with  a  resulting  F  ratio  »  4.96. 

Only  use  data  points  that  have  multiple  replications,  in  this  case  n(cp). 

4.3.  Multicollinearity  Test. 

The  terms  collinearity,  multicollinearity  and  near  collinearity  refer  to  correlations  among 
variables  where  the  two  variables  are  perfectly  correlated.  This  can  render  the  regression 
statistics  on  the  variables  to  be  useless  (Pedhazur,  1997).  For  this  reason  the  first  step  in  looking 
at  the  output  data  is  to  check  for  this  condition.  A  correlation  table  showing  the  correlation  of 
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the  dependent  variables  to  the  independent  variables  was  generated.  (The  SAS  program  code 
and  output  used  to  generate  this  table  are  at  appendix  E.)  The  correlation  of  the  dependent 
variables  against  the  independent  variables  from  this  run  is  shown  in  the  correlation  matrix  in 
^  Table  1 8.  The  top  number  for  each  correlation  shows  ^  and  the  bottom  number  shows  the 

resulting  P  value.  Those  correlations  that  show  a  significant  P  value  correlation  at  the  .01  level 
are  highlighted  with  a  box  drawn  around  them.  The  dependent  measures  TLTT,  TLKSA,  UTIL, 
*  NOQUE,  TSKINT  and  TSKDRP  all  show  a  significant  correlation  to  the  independent  variable 

USP  at  the  .01  level.  These  six  dependent  measines  cluster  together  into  a  performance  group 
that  will  be  called  ‘Forming  a  Mental  Model  (FMM)’.  The  performance  task  FMM  is  therefore 
indicated  to  be  the  primary  attentional  focus  of  the  battalion  commander  and  is  the  conglomerate 
of  activities  that  should  be  allocated  the  most  attention  when  trying  to  improve  the  ability  to 
perform  the  required  job  fimctions.  NOQUE  also  shows  significance  to  the  CAR  IV.  ThelV’s 
DRD  and  EEI  do  not  show  any  correlation  significance  to  any  of  the  dependent  measures 
indicating  that  the  task  activities  of  making  decisions  and  evaluating  what  the  impact  of  those 
decisions  and  actions  might  be  do  not  significantly  occupy  the  battalion  commander’s  time.  To 
be  sure,  making  decisions  and  directing  that  the  decisions  be  carried  out  is  a  part  of  the  battalion 
commander’s  primary  duty  in  the  real  world.  However,  what  CoHOST  is  intimating  is  that 
making  decisions  could  well  be  almost  automatic  after  weighing  all  the  alternatives  formed  from 
achieving  a  complete  tmderstanding  of  the  situational  picture,  and  that  it  occurs  without 
significant  mental  effort  on  the  part  of  the  battalion  commander. 

Table  18  -  Correlation  Matrix  For  Dependent  And  Independent  Variables 


Pearson  Correlation  Coefficients,  N  »  19 
Prob  >  |r|  under  HO:  Rho=0 


tltt  I 

tlksa  1 

util 

noque 

tskint  1 

tsksup  1 

tskdrp 

0.03460 

-0.03310 

-0.47420 

0.64551 

-0.11051 

-0.15129 

0.47411 

0.8882 

0.8930 

0.0402 

0.0028 

0.6524 

0.5364 

0.0403 

0.31714 

-0.05695 

-0.43205 

0.09847 

-0.23848 

-0.42823 

0.22753 

0.1858 

0.8169 

0.0647 

0.6884 

0.3255 

0.0674 

0.3489 

-0.20149 

0.02052 

-0,10538 

0.33917 

-0.35480 

-0.12052 

0.22753 

0.4081 

0.9336 

0.6677 

0.1555 

0.1361 

0.6231 

0.3489 

-0.61284 

0.99078 

-0.60066 

0.64934 

-0.70379 

-0.25386 

0.76122 

0.0053 

<.0001 

0.0065 

0.0026 

0.0008 

0.2943 

<.0001 
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4.4.  Deteimination  of  Polynomial  Regression  Models. 

The  goal  of  this  part  of  the  analysis  is  to  determine  the  optimum  polynomial  regression 
models  that  describes  the  performance  characteristics  of  each  dependent  variable.  However, 
since  this  is  a  2^  experimental  design  the  polynomial  regression  equations  are  equivalent  to  the 
multiple  linear  regression  case.  The  multiple  linear  regression  model  to  be  used  in  this  thesis 
was  determined  by  developing  regression  parameter  estimates  (beta  weights)  for  each  dependent 
variable.  Using  the  standardized  database,  the  MRS  independent  variable  for  each  dependent 
variable  is  examined  to  identify  the  smallest  p  value.  This  value  is  then  xised  as  a  threshold  to 
determine  which  of  the  other  coefficients  are  significant  and  whose  value  should  be  included  in 
the  regression  model  for  that  dependent  variable.  (The  SAS  program  code  and  run  results  used 
to  generate  these  regression  coefficients  is  at  appendix  F.)  The  MRS  ‘p’  values  is  shown  in 
Table  19.  The  smallest  P  value  is  .0209  for  the  dependent  variable  UTIL  highlighted  with  an 
asterisk.  This  value  is  therefore  used  as  the  threshold  for  significance  to  determine  which  of  the 
independent  variables  will  be  used  in  the  regression  equation  for  each  dependent  variable. 

Table  19  -  Standardized  P  Values  For  The  MRS  IV  For  Each  DV 


Variable 

DF 

Parameter  Estimates 
Parameter  Standard 

Estimate  Error 

t  Value 

Pr  >  |t| 

mrs 

1 

Dependent  Variable:  tltt 
0.46996  0.25776 

1.82 

0,0913 

mrs 

1 

Dependent  Variable:  tlksa 
-0.08609  0.04221 

-2.04 

0.0623 

mrs 

1 

Dependent  Variable:  util 
-0.41098  0.15650 

-2.63 

0.0209* 

mrs 

1 

Dependent  Variable:  noque 
-0.13047  0.07131 

-1.83 

0.0903 

mrs 

1 

Dependent  Variable:  tskint 
0.16577  0.22691 

0.73 

0.4780 

mrs 

1 

Dependent  Variable:  tsksup 
-0.33464  0.33925 

-0.99 

0.3419 

mrs 

1 

Dependent  Variable:  tskdrp 
0.07229  0.10110 

0.72 

0.4872 
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The  parameter  estimates  (beta  weights)  for  each  dependent  measure  with  a  p  value  greater  than 
.0209  was  determined  by  evaluation  of  the  statistical  analysis  in  Appendix  F  and  is  shown  in 
Table  20. 

Table  20  -  IV  Regression  Coefficients  For  The  Standardized  Data 


IV  Title 

Comm  & 
Report 

Decide  & 
Recommend  / 
Direct 

Evaluate  & 
Estimate 
Impact 

Identify/ 

Understand 

Situational 

Picture 

DV  Title 

Variable  Code 

CAR 

DRD 

EEI 

USP 

TLTT 

-0.919 

TLKSA 

1.486 

0.000 

utilization 

-0.711 

-0.648 

mmm 

0.000 

Number  of  Queues 

0.968 

0.509 

mMEM 

0.000 

^m\hm 

IBBHi 

lljllllllllllllllllll 

IIIIMHIIII 

0.000 

TSKSUP 

■ebhi 

TSKDRP 

msssm 

0.341 

0.341 

1.172 

0.000  1 

Expressing  these  coefficients  as  regression  equations  generates  an  expression  for  the  response  of 
each  dependent  variable  to  each  independent  variable.  These  expressions  are  shown  at  Equations  4. 


TLTT 

= 

-0.919USP 

TLKSA 

= 

+1.486USP 

UTIL 

=  -0.71  ICAR 

-0.648DRD 

-0.901USP 

NOQUE 

=  0.96826CAR 

+  0.50875EEI  +0.974USP 

TSKINT 

= 

+1.056USP 

TSKSUP 

TSKDRP 

=  0 

=  0.71  ICAR 

+0.341DRD 

+0.341EEI  +1.172USP 

Equations  4  -  Regression  Equations  From  Standardized  Data 


4.5.  Determination  of  Maximum  Values  Of  Response  Surfaces. 

In  order  to  determine  the  maximum  value  of  the  response  surface  for  each  dependent 
variable,  these  equations  are  set  equal  to  ‘O’,  the  partial  differential  taken  for  each  independent 
variable,  and  the  resulting  equation  sets  solved  to  provide  the  values.  Using  the  dependent 
variable  TSKDRP  as  an  example,  its  regression  equation  is: 


TSKDRP  =0.71  ICAR 


■H).341DRD  +0.341EEI  +1.172USP 


Taking  the  partial  derivative  of  each  independent  variable  gives: 

TSKDRP’car 

=  0.711 

+0.341DRD 

+0.341EEI 

+1.172USP 

TSKDRP’drd 

=  0.71  ICAR 

+0.341 

+0.341EEI 

+1.172USP 

TSKDRP’eei 

=  0.71  ICAR 

+0.341DRD 

+0.341 

+1.172USP 

TSKDRP’usp 

=  0.71  ICAR 

+0.341DRD 

+0.341EEI 

+1.172 

Setting  each  partial  derivative  equal 

to  zero  gives: 

0.711 

+0.341DRD 

+0.341EEI 

+1.172USP 

=  0 

0.71  ICAR 

+0.341 

+0.341EEI 

+1.172USP 

=  0 

0.71  ICAR 

+0.341DRD 

+0.341 

+1.172USP 

=  0 

0.71  ICAR 

+0.341DRD 

+0.341EEI 

+1.172 

=  0 

Expressing  these  equations  in  matrix  form  as  shown  in  Figure  1 1  and  solving  (MathSoft,  2001) 
for  the  4  unknowns  gives: 

Taskload  From  Number  of  Tasks  Dropped  -  TSKDRP 


^  0  0.34129  0.34129  1.17183^ 

^-0.71117^ 

0.71117  0  0.34129  1.17183 

-0.34129 

0.71117  0.34129  0  1.17183 

V  1= 

-0.34129 

^0.71117  0.34129  0.34129  0  > 

^-1.17183; 

soln  :=  lsolve(M,v) 


(  -0.203') 


soln  = 


-1.506 

-1.506 


V  0.27  ) 


Figure  11  -  Determination  of  Maximum  Regression  Values 


Expressing  the  results  back  into  equation  form  gives  the  maximum  value  in  the  response  surface 
for  each  independent  measure  for  the  dependent  measure  TSKDRP: 


CAR 

=  -0.203 

DRD 

=  -1.506 

EEI 

=  -1.506 

USP 

=  0.27 

The  solutions  for  all  the  dependent  variables  is  at  appendix  G.  Table  21  shows  the  maximum 
response  siuface  value  for  each  of  the  dependent  measures. 
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Table  21  -  Response  Surface  Maximum  Values  For  Standardized  Data 


standardized  Data 

IV  Title 

Comm  & 
Report 

Decide  & 
Recommend  / 
Direct 

Evaluate  & 
Estimate 
Impact 

Identify  / 
Understand 
Situationai 
Picture 

DV  Title 

Variable  Code 

CAR 

DRD 

EEI 

USP 

Taskload  [Task  Tirnel 

TLTT 

-0.919 

Taskload  [KSAI 

TLKSA 

1.486 

Utilization 

UTIL 

-0.589 

-0.744 

-0.254 

Number  of  Queues 

NOQUE 

-0.503 

-0.957 

-0.022 

Tasks  Interrupted 

TSKINT 

1.056 

Tasks  Suspended 

■iMIfiilJi 

Tasks  Dropped 

-0.203 

-1.506 

-1.506 

0.270 

This  data  is  shown  graphically  in  Figure  12. 


4 


Dependent  Vaiiablee 


■  CAR 

■  DRD 

□  EEI 

□  USP 


Independent 

Variables 


Figure  12  -  Response  Surface  Maximum  Values  For  Each  IV  To  Each  DV 
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4.6.  Significance  Test  Of  Each  DV  To  Each  IV. 


From  the  Regression  Model  ANOVA  summary  table  design  presented  in  Table  17,  the 
results  of  statistical  tests  performed  to  establish  the  significance  of  each  dependent  variable  to 
€  each  independent  variable  is  presented  in  Regression  ANOVA  tables  in  Appendix  I.  The 

determination  of  significance  for  LOF  is  determined  by  a  manual  evaluation  using  a  decision  rule 
and  comparing  a  tabled  F  ratio  to  the  observed  F  ratio  as  calculated.  These  LOF  calculations  are 

Hi 

also  at  Appendix  I  and  is  summarized  in  Table  22.  An  example  of  a  Regression  ANOVA  table  is 
shown  for  the  dependent  variable  TLTT  in  Table  23. 

Table  22  -  Siumnary  of  LOF  Tests  of  Significance 


LOF  Significance  Tests 

F  Ratio 

Value 

Significant  ? 

Ftabled-LOF”  ^(3,10)  = 

6.550 

Pobserved-LOF-TLTT  = 

13.952 

Yes 

Fobserved-LOF-TLKSA  “ 

16.644 

Yes 

Fobserved-LOF-Um  = 

12.872 

Yes 

Pobserved-LOF-NOQUE  ~ 

2.761 

No 

Fobserved-LOF-TSKINT  ” 

11.782 

Yes 

Fobscrved-LOF-TSKSUP  “ 

38.297 

Yes 

Pobserved-LOF-TSKDRP  = 

9.551 

Yes 

$ 
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Table  23  -  Regression  ANOVA  Simunary  Table  For  Dependent  Variable  TLTT 


Regression  ANOVA  Summary  Table  | 

Augmented  1-4  Replicate  2-5  Design 

Alternative  Regression  ANOVA 

Number  of  center  point  treatments,  n(cp)  = 

11 

Number  of  Data  Points  That  Have  Repeated  Observations 

(T)  (i.e.,  the  replication  of  the  Center  Point)  = 

1 

Treatment  Com 

>inations,  N  (n(l/4)  +  n(cp)xT)  = 

19 

Source 

df 

MS 

F 

P 

Regression 

(5) 

CAR 

1 

0.010 

0.160 

0.696 

DRD 

1 

0.877 

13.590 

0.004 

EEI 

1 

0.354 

5.480 

0.041 

USP 

1 

3.276 

50.730 

<.0001 

MRS 

1 

0.856 

13.260 

0.005 

Residuals 

(13) 

UckOfFit(LOF) 

3 

0.901 

13.952 

Error  ((n(cp)  -1)  x  T) 

10 

0.065 

Total: 

(18) 

Verification:  N-1  = 

18 

*  From  manual  decision  rule  test. 


4.7.  Identification  of  Higher  Order  Components  For  Main  Effects. 

Following  the  identification  of  LOF  significance  comes  the  task  of  identifying  what  the 
higher  order  response  looks  like  for  those  variables  found  to  have  a  significant  LOF.  One  caveat, 
however,  is  that  in  human  factors  work  it  is  rarely  possible  to  interpret  interactions  and  responses 
higher  than  the  second  order.  Thus,  the  effort  here  will  focus  on  identifying  only  the  quadratic 
responses  and  in  attempting  to  determine  what  their  response  surface  looks  like. 

After  determining  which  dependent  variables  are  predicted  to  have  a  quadratic  response 
to  at  least  one  of  the  independent  variables,  the  second  step  is  to  identify  which  second  order 
effects  are  significant.  Because  this  experimental  design  only  has  3  treatment  levels  resulting 
from  the  augmented  ‘A  replicate  fictional  factorial  design,  there  is  not  enough  data  to  resolve  the 
quadratic  components  of  the  main  effects.  An  alternate  manual  approach  is  therefore  taken 
where  a  2  dimensional  response  plot  of  each  dependent  variable  against  each  independent 
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variable  is  generated  in  EXCEL.  Subjective  interpretation  of  these  plots  then  establishes  the 
response  order  for  each  of  the  dependent  measures.  Each  dependent  variable  that  has  been 
shown  to  be  contributing  to  the  response  surface  in  at  least  a  quadratic  form  as  shown  by  the 
^  LOF  test  results  in  Table  23  needs  to  be  examined  to  see  which  of  the  independent  variables  it  is 

generating  a  quadratic  response  to.  Because  of  the  nature  of  the  experimental  design  these 
results  caimot  be  obtained  directly  from  response  surface  regression  runs.  This  is  due  to  the  fact 
t  that  the  design  being  used  can  be  considered  an  incomplete  central  composite  design  (CCD) 

where  the  fractional  factorial  data  points  are  collocated  with  what  would  otherwise  be  the  face 
centered  data  points  in  a  true  CCD.  Thus,  attempts  to  generate  the  quadratic  results  for  the  3 
levels  in  the  design  multiply  the  +1  and  +1  treatments  and  the  -1  and  -1  treatments  and  arrives  at 
+1  for  both  conditions  thereby  reducing  the  munber  of  treatments  for  the  quadratic  analysis  from 
3  to  2.  As  one  more  treatment  level  than  the  order  of  the  analysis  is  required  to  perform  the  test, 
anything  higher  than  a  linear  analysis  is  not  possible.  A  true  CCD  would  have  5  treatment  levels 
with  the  a  values  being  away  from  the  face  centered  levels  of  the  fractional  factorial  data  points 
and  would  allow  at  least  a  quadratic  resolution  of  the  results. 

This  information  can  still  be  obtained,  however,  from  manually  interpreting  a  graph  of 
the  response  for  each  dependent  variable  for  each  dependent  variable.  The  data  for  each 
treatment  level  for  each  dependent  variable  is  averaged  to  provide  a  response  value  for  that 
dependent  variable  against  each  independent  variable.  This  provides  three  data  points  for  each 
condition  that  can  be  plotted  to  display  the  response  characteristic  of  that  treatment  for  the 
IV/DV  combination.  Using  EXCEL  these  plots  were  generated  using  the  curve  smoothing 
option  to  produce  the  response  line  for  the  dependent  measure.  These  plots  are  then  examined  to 
see  which  dependent  measure  is  demonstrating  a  quadratic  response.  The  data  matrix  of  the  data 
*  generated  by  averaging  the  treatment  data  is  in  Table  24  and  a  sample  of  these  plots  is  shown  in 

Figure  1 3.  A  complete  printing  of  all  the  plots  is  in  appendix  J. 
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Table  24  -  Average  of  Treatment  Data  For  Each  IV/DV  Combination 


tItt/CAR 

tIksa/CAR 

util/CAR 

noque/CAR 

tskint/CAR 

tsksup/CAR 

tskdrp/CAR 

Average  of  Sum  of  -1  data 

-1 

-1.65591 

-0.00504 

1.62394 

-3.52635 

-1.27595 

-1.84628 

-1.92264 

Average  of  Sum  of  0  data 

0 

0.26333 

0.03703 

0.22205 

-0.06304 

0.35255 

0.50073 

-0.16764 

Average  of  Sum  of  1  data 

1 

-1.24072 

-0.40224 

-4.06650 

4.21977 

-2.60211 

-3.66178 

9.57071 

tItt/DRD 

tlksa/DRD 

noque/DRD 

tskint/DRD 

tsksup/DRD 

tskdrp/DRD 

Average  of  Sum  of  -1  data 

-1 

-3.35114 

0.13808 

1.37103 

-0.24410 

-0.50818 

-0.18463 

5.36087 

Average  of  Sum  of  0  data 

0 

0.26333 

0.03703 

0.22205 

-0.06304 

0.35255 

0.50073 

-0.16764 

Average  of  Sum  of  1  data 

1 

0.45451 

-0.54536 

-3.81359 

0.93751 

-3.36988 

-5.32343 

2.28720 

tltt/EEl 

tIksa/EEi 

util/EEl 

noque/EEl 

tskint/EEl 

tsksup/EEi 

tskdrp/EEl 

Average  of  Sum  of  -1  data 


Average  of  Sum  of  0  data 
Average  of  Sum  of  1  data 


,  0.26333  0.03703 
1  -2.65725  ■0.08054 
tItt/USP  tIksa/USP 


2.22875  -6.14834 


0.26333  0.03703 


-5.12538  5.74106 


0.58901 


0.22205 


-1.85355 


util/USP 


2.38266 


0.22205 

-4.82522 


-1.68829 


-0.06304 

2.38171 

noque/USP 


-3.54932 


-0.06304 

4.24274 


0.18980 


0.35255 

-4.06786 

tskint/USP 


2.28373 


0.35255 

-6.16179 


-2.03090 


0.50073 

■3.47715 

tsksup/USP 


-1.23085 


0.50073 

-4.27720 


5.36087 


-0.16764 

2.28720 

tskdrp/USP 


2.03873 


-0.16764 

5.60934 


tIksa/CAR 


0.10000 

0.00000 

-0.10000 

-0.20000 

-0.30000 

-0.40000 

-0.50000 


8.00000 

6.00000 

4.00000 

2.00000 

0.00000 

-2.00000 

•4.00000 

-6.00000 

-8.00000 


tIksa/USP 


Figure  13  -  Independent  Variable  Response  Plots  For  Dependent  Variable  TLKSA 
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Table  25  expands  the  previous  LOF  significance  test  table  to  show  the  order  of  the  probable 
response  surface  effects  from  the  subjective  evaluation  of  the  plots. 

Table  25  -  Sununary  of  Interpretation  of  Response  Order  For  The  DV’s 


Interpretations  of  Response  Surface  Plots 

1  F  Ratio 

Value 

ma 

Order  of  Response 

6.550 

CAR 

DRD 

EEI 

USP 

HHEBSlSSiS 

13.952 

Yes 

2 

2 

2 

1-2 

16.644 

Yes 

1-2 

1-2 

2 

1 

HHESSHSS 

12.872 

Yes 

1 

1 

2 

1 

2.761 

No 

1 

1 

1 

1 

11.782 

Yes 

2 

2 

1-2 

1-2 

38.297 

Yes 

2 

2 

2 

2 

9.551 

Yes 

1-2 

2 

2 

2 

Legend: 

2  -  Indicates  probable  quadratic  response. 

1  -  Indicates  probable  linear  response. 

1-2  -  Curve  could  be  either  linear  or  quadratic. 


All  of  the  interpretations  of  these  response  curves  involve  subjective  opinion  as  to  the 
nature  of  the  curve.  Those  that  are  coded  as  quadratic  indicate  enough  variability  to  be  probably 
interpreted  as  quadratic.  However,  only  one  of  the  curves  coded  as  linear  showed  clearly  a 
straight  line  response.  This  curve  was  for  the  TLKSA  vs.  USP  response.  All  of  the  rest  of  the 
curves  interpreted  as  linear  showed  a  minimum  amount  of  curve  in  the  line  which  was 
attritibuted  to  random  variability  in  the  model  and  were  interpreted  as  probably  linear.  All  of  the 
curves  coded  as  ‘1-2’  are  probably  linear,  however,  subjective  assessment  can  not  state  with 
conq)lete  confidence  that  there  is  no  quadratic  component  in  the  response. 


5.  Discussion. 


The  development  of  methodologies  to  capitalize  on  the  use  of  computer  simulations  to 
support  empirical  research  begins  with  a  step  by  step  look  first  at  statistical  procedures  that  are 
applicable  to  the  type  of  experimental  designs  that  might  be  used.  Noting  that  numerical  results 
alone  are  often  hard  to  understand  by  clients  for  whom  the  research  is  being  conducted,  the 
second  phase  in  the  methodology  is  to  interpret  the  results  in  terms  of  the  work  situation  that  the 
simulation  is  describing.  The  approach  here  is  to  invoke  accepted  human  performance  models 
from  teamwork  and  macroergonomic  literature  to  provide  a  framework  for  translating  the  results 
of  the  research  into  terminology  that  these  individuals  can  accept  and  utilize.  This  thesis  uses  the 
CoHOST  simulation  as  a  platform  to  illustrate  the  development  of  these  methodologies.  In  the 
process  of  this  effort  conclusions  and  recommendations  are  produced  from  CoHOST  that  are  of 
interest  to  the  original  Army  CoHOST  clients. 

This  section  begins  with  descriptions  of  the  statistics  involved  in  the  study  that  include 
correlations,  development  of  regression  equations,  and  attempts  to  determine  the  existence  of 
higher  order  effects  in  the  dependent  variables.  A  team  performance  model  is  then  used  to 
provide  a  basis  for  the  explanation  of  the  statistical  results.  The  major  focus,  however,  of  the 
discussion  are  descriptions  of  the  methodology  as  a  whole  and  how  the  CoHOST  model  was 
used  both  to  develop  the  methods  and  then  provide  an  example  of  their  use.  Implications  of  the 
work  are  discussed  along  with  discussions  of  where  the  research  can  go  from  here  and 
suggestions  for  future  work. 

5.1.  Predictions  From  Correlation  Table. 

The  interpretation  of  Table  18  provides  insights  into  what  this  simulation  model  is 
predicting  for  the  activities  of  the  battalion  commander.  These  results  indicate  that  the  battalion 
commander  is  not  spending  any  significant  amount  of  time  or  devotion  of  mental  and  physical 
resomces  to  either  the  activities  of  Deciding  and  Recommending  or  Directing  (DRD)  or 
Evaluating  and  Estimating  Impact  (EEI).  Referring  to  the  nature  of  activities  in  a  battalion 
tactical  operations  center  (TOC)  and  the  fact  that  the  battalion  conunander  spends  the  majority  of 
the  available  time  displaced  forward  of  the  TOC  roaming  the  front  line  in  a  command  and 
control  vehicle,  it  is  consistent  with  observed  real  world  activities  that  these  types  of  duties 
would  be  relegated  to  subordinates,  particularly  the  executive  officer  and  the  battle  captain.  The 
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dependent  variable  ‘Number  of  Times  a  Task  Was  Suspended  (TSKSUP)  did  not  correlate  with 
any  of  the  independent  variables  thereby  suggesting  that  intermpted  tasks  were  more  likely  to  be 
eventually  dropped  rather  than  being  held  in  a  suspended  status  for  eventual  interruption.  As  this 
^  computer  model  simulates  activities  under  actual  combat  conditions  where  the  time  and  stress 

pressures  are  high,  this  is  a  consistent  conclusion  where  the  battalion  commander  is 
predominately  focused  on  those  activities  requiring  immediate  attention  and  is  referring  less  time 
«  sensitive  issues  to  subordinates. 

There  are  two  dependent  measures  that  evaluate  the  effects  of  task  load  on  this  operator. 
The  ‘Taskload  from  Task  Time  (TLTT)’  measure  looks  at  the  amount  of  taskload  imposed  on  the 
battalion  commander  as  a  result  of  how  long  it  takes  to  perform  individual  tasks.  The  ‘Taskload 
from  KSA  (TLKSA)’  measure  looks  at  the  taskload  imposed  as  a  result  of  the  amoimt  of 
knowledge,  skills,  and  abilities,  primarily  cognitive  and  psychomotor,  that  the  battalion 
commander  must  allocate  to  the  task  in  order  to  satisfy  it.  Both  of  these  measures  significantly 
correlate  only  to  the  independent  measure  ‘Understand  Situational  Picture  (USP)’.  The 
dependent  measure  ‘Number  of  Times  a  Task  Was  Interrupted  (TSKINT)  also  only  significantly 
correlates  to  USP.  When  these  activities  are  then  combined  with  the  dependent  measure 
Utilization  (UTIL),  a  picture  begins  to  emerge  of  what  the  correlation  statistics  are  predicting  to 
be  the  primary  focus  of  the  battalion  commander’s  activities. 

This  focus  is  a  predominate  amoimt  of  attention  and  activities  to  developing  and 
maintaining  a  clear  understanding  of  the  evolving  battlefield  situation  and  then  when  necessary 
communicating  that  knowledge  along  with  directives  and  orders  back  to  the  TOC  and  the 
company  commanders  being  directed.  These  activities  consume  the  battalion  commander’s 
focus  and  attention  to  the  point  that  the  operator  is  constantly  being  intermpted  by  changes  in  the 
situation  he  or  she  is  trying  to  maintain  an  awareness  of  and  possibly  by  communication  and 
coordination  queries  from  subordinates  as  well  as  from  contemporaries  and  superiors. 

^  The  other  dependent  measures  that  significantly  correlate  to  the  independent  variables  are 

‘Number  of  Queues  Generated  During  the  Run  (NOQUE)’  and  ‘Number  of  Tasks  That  Were 
Dropped  During  the  Run  (TSKDRP)’.  This  is  consistent  with  the  above  assessment  that  there 
are  many  task  based  demands  that  are  placed  on  the  battalion  commander  that  have  to  wait  their 
turn  for  attention  and  the  fact  that  many  of  them  time  out  and  are  considered  to  be  dropped  as 
they  never  get  processed.  The  simulation  tracks  the  timing  of  each  task  and  considers  that  if  a 
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task  is  not  perform  within  a  specified  period  of  time  then  the  information  and  requirements  of 
that  task  have  become  obsolete  and  the  task  is  therefore  dropped. 

5.2.  Regression  Equation  Results. 

Plots  of  the  response  surface  maximum  values  in  Figure  12,  supported  by  the  table  of 
regression  coefficients  in  Table  20,  contribute  to  an  evolving  picture  of  what  is  generating  the 
highest  demands  on  the  battalion  commander.  The  regression  coefficients  closely  resemble  the 
correlation  results  that  show  a  strong  dominance  of  activities  directed  toward  performance  of  the 
understand  situational  picture  (USP)  independent  variable.  Also  coinciding  between  the  two 
results  is  the  strong  response  of  the  dependent  variable  TSKDRP  (number  of  tasks  dropped)  that 
significantly  responds  to  all  the  independent  variables.  Thus,  the  conunents  of  the  preceding 
section  are  echoed  with  the  regression  solutions  that  show  the  battalion  commander 
predominately  focused  on  tasks  associated  with  understanding  the  evolving  battlefield  situation 
as  it  is  perceived  through  the  observation  and  communication  means  that  is  available.  The 
primary  secondary  focus  is  communication  based  activities  to  issue  and  receive  directives.  As 
new,  immediate  priority  issues  attract  the  battalion  commander’s  attention,  it  is  most  likely  that 
the  tasks  being  currently  performed  will  be  interrupted  to  attend  to  the  new  high  priority  issue 
which  means  that  the  tasks  crirrently  being  performed,  once  interrupted,  are  likely  to  become 
obsolete  by  virtue  of  timing  out  and  will  therefore  get  dropped  and  never  completed. 

5.3.  Estimates  Of  Higher  Order  Effects. 

As  previously  described,  the  assessment  of  quadratic  effects  in  the  responses  of  the 
dependent  variables  to  the  main  effects  has  been  a  highly  subjective  process  in  this  study. 
Because  of  the  nature  of  the  experimental  design  that  was  optimized  for  data  gathering  efficiency 
there  is  simply  not  enough  data  to  support  direct  calculation  of  the  qriadratic  effects  using 
statistical  procedures  with  tools  such  as  SAS.  It  was  not  realized  until  late  in  the  analysis  process 
that  these  limitations  would  become  a  concern,  however,  it  points  out  the  adage  that  ‘nothing 
comes  for  free’.  If  extended  time  had  been  available  to  gather  a  full  ‘A  or  even  foil  %  replicate 
fractional  factorial  number  of  treatments  or  a  foil  CCD  worth  of  treatments  there  would  have 
been  enough  data  points  to  support  quadratic  assessments  of  the  main  effects. 

However,  without  the  necessary  data  for  actual  qualitative  results  the  study  is  relegated  to 
a  subjective  examination  of  the  results  that  are  obtainable  from  the  data.  The  plots  showing  the 
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responses  of  the  dependent  variables  to  each  independent  variable  along  the  3  data  points 
resulting  from  the  3  treatment  combinations  at  least  provides  clues  as  to  what  is  occurring  in  the 
model.  Referring  to  these  plots  in  appendix  J  and  the  interpretation  of  their  meaning  in  Table  26, 
^  predictions  can  be  made  as  to  the  probability  of  higher  order  actions  occurring  in  the  simulation. 

The  general  criteria  used  in  the  interpretation  of  the  plots  was  that  if  the  curves  crossed  the  x  axis 
twice  the  interaction  probably  contains  some  quadratic  effect.  Because  of  the  previous  attention 
«  to  the  TSKDRP  dependent  variable  and  the  USP  independent  variable  the  relationship  between 

them  is  considered  first.  By  the  criteria  above  this  response  shows  a  probable  quadratic  effect. 

In  performance  terms  this  indicates  that  the  battalion  commander  is  regarding  the  battlefield 
situational  assessments  with  greater  and  lesser  amounts  of  attention  and  time  according  to  the 
number  and  severity  of  the  events  being  observed.  The  results  of  this  activity  would  be  a 
changing  number  of  existing  tasks  not  being  completed,  or  becoming  dropped,  as  attentional 
focus  is  constantly  being  changed  while  trying  to  maintain  an  accurate  mental  model  of  the 
battlefield.  Conversely,  the  model  output  seems  to  show  that  the  remainder  of  the  activity 
response  is  probably  linear  in  nature  with  a  straight  line  proportional  response  to  the  demands  of 
the  other  task  types. 

5.4.  Mapping  A  Macroergonomic  Model  To  The  CoHOST  Predictions. 

After  the  predicted  human  performance  and  environmental  domain  characteristics  have 
been  determined  from  the  above  analysis,  it  is  important  to  lay  aside  the  simulation  and  the 
experimental  design  manipulations  and  ask  the  question,  “what  does  it  mean?”  in  the  context  of 
real  world  performance  issues.  This  query  may  be  addressed  according  to  a  structured  approach 
such  as  can  be  found  from  a  number  of  macroergonomic  framework  models.  A  teamwork  model 
^  that  is  particularly  suited  for  this  purpose  is  shown  in  Figure  1 4  that  “examines  team  building 

and  its  influence  on  team  effectiveness”  (Swezey  and  Llaneras,  1997;  Tarmenbaum,  Beard,  and 
Salas,  1992).  This  model,  called  the  team  effectiveness  model  (TEM),  integrates  the  aspects  of 
*  several  models  that  were  developed  to  explain  team  processes  and  outcomes  (Weaver,  Bowers, 

Salas,  and  Carmon-Bowers,  1995).  Adapting  Tannenbaum’s  definition  of  what  a  team  is  into  the 
military  command  and  control  domain  defines  a  team  as  two  or  more  people  who  are  interacting 
toward  a  common  mission.  Similar  definitions  abound  in  the  literature  (Weaver  et  al.,  1995). 
The  23  individuals  fiom  the  1000  member  battalion  task  force  who  are  engaged  in  command  and 
control  operations  in  the  battalion  tactical  operations  center  (TOC)  clearly  meet  this  definition  of 
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a  functioning  team.  The  battalion  commander  is,  without  question,  the  leader  of  this  team 
although  this  person  typically  is  not  collocated  with  most  of  the  team  members  during  combat 
operations.  Attempts  to  improve  the  functioning  of  a  team,  often  called  team  building,  generally 
focus  on  improving  team  operations  by  doing  such  things  as  removing  barriers  to  effective 
operations  and  clarifying  roles  of  the  individual  team  members. 


Figure  14  -  Team  Effectiveness  Model 
(Tannenbaum  et  al.,  1992) 


The  basic  nature  of  the  TEM  model  follows  the  general  systems  theory  paradigm  of 
input-throughput-output  much  the  same  as  the  basic  structure  of  the  CoHOST  model  itself 
Among  other  things,  this  model  describes  team  inputs  as  the  characteristics  of  the  tasks  presented 
to  the  team.  This  coincides  with  the  input  structure  to  the  CoHOST  model  with  task 
performance  generating  communication  messages  arriving  to  the  workgroup  from  higher,  lower, 
and  adjacent  organizational  elements.  Throughputs  in  the  Tannenbaum  model  are  described  as 
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the  maimer  in  which  the  team  performs  its  interactions  over  time.  These  are  words  which  can 
identically  be  used  to  describe  the  throughput  functioning  of  the  CoHOST  simulation.  Finally, 
the  outputs  from  the  TEM  model  are  primarily  the  quality  and  quantity  of  the  products  produced 
by  the  team.  In  the  CoHOST  model  this  translates  to  the  decisions  made  by  the  team,  primarily 
the  battalion  commander,  and  the  conununication  of  these  decisions  to  other  organizational 
elements  outside  the  team.  There  are  some  characteristics  of  the  Tannenbaum  model  that  are  not 
reflected  in  CoHOST.  For  example,  Taimenbaum  considers  that  there  are  other  outputs  in  the 
team  functioning  beyond  just  the  primary  output  of  team  performance.  Such  things  as  changes  in 
the  team  makeup  or  organization  resulting  in  new  roles  and  interactions  and  a  greater  or  lesser 
cohesiveness  to  be  team  outputs  as  well.  The  CoHOST  model  limits  itself  to  strictly  looking  at 
the  resulting  performance  of  the  team  as  the  measure  of  team  output. 

One  advantage  of  using  a  simulation  model  such  as  CoHOST  over  live  subjects  with 
empirical  observation  is  that  exact  repeatability  can  be  obtained  that  transcends  the  normal 
fluctuations  that  can  occur  in  team  performance  due  to  human  interpersonal  differences  in  skill 
levels,  opinion,  dedication,  and  attention  to  detail,  for  example.  While  individual  characteristics 
of  different  operators  performing  the  same  job  can  be  approximated  with  different  sequences  of 
random  variability,  the  simulated  operators  can  be  held  as  a  constant  while  varying  the  demands 
of  the  work  domain.  CoHOST  provides  this  performance  medium,  within  the  constraints  of  its 
simulated  battlefield  scenario  and  the  taxonomically  described  capabilities  of  each  operator, 
where  the  demands  and  task  performance  abilities  of  one  operator,  such  as  the  battalion 
commander,  can  be  looked  at  in  isolation  but  yet  within  the  performance  constraints  of  the 
overall  team.  To  be  certain,  there  are  limitations  in  a  simulation  based  approach  to  evaluate 
individual  and  team  decision  making  and  performance.  It  is  noted  (Cannon-Bowers,  Salas,  and 
Pruitt,  1996)  that  it  is  not  possible  to  apply  empirically  based  decision  making  analyses  to  many 
real  world  situations  because  these  approaches  do  not  account  for  the  decision  maker’s 
experience  nor  the  con^lexity  of  the  task  or  the  true  demands  of  the  environment.  Also,  there 
must  be  a  full  understanding  of  what  the  simulation  is  and  is  not  addressing  and  there  is  a 
continuing  need  for  researchers  to  ‘systematically  test  the  components  of  the  model  in  order  to 
determine  their  relative  importance’  (Weaver  et  al.,  1995). 

Analyses  performed  in  previous  sections  have  established  some  measure  of  the  primary 
attention  focus  of  the  battalion  commander.  It  must  be  reemphasized  that  these  conclusions  and 
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the  results  of  this  experimental  observation  are  from  the  world  according  to  CoHOST.  One  of 
the  primary  assumptions  made  before  the  study  began  was  that  there  was  a  computer  simulation 
model  available  that  accurately  (i.e.,  is  believable  to  a  specified  performance  interval  and  that  the 
risk  of  error  involved  in  using  it  is  within  acceptable  limits)  postulated  a  particular  human 
performance  domain.  The  name  of  the  model  is  CoHOST  and  the  performance  domain  is  the 
command  and  control  work  group  in  a  U.S.  Army  battalion  level  tactical  operations  center 
conducting  combat  operations.  Whatever  the  level  of  risk  assessed  to  be  associated  with  the  use 
of  the  model  also  accompanies  the  reliability  of  the  results  generated  by  it.  Before  this  thesis 
study  began  a  formal  risk  assessment  according  to  the  Failure  Modes  and  Effects  Analysis 
(FMEA)  procedure  (Croxall  and  Wood,  1989;  Goddard  and  Davis,  1984;  Knepell  and  Arangno, 
1993)  established  that  the  risk  associated  with  die  use  of  this  simulation  was  within  acceptable 
limits  for  the,  then,  current  study. 

Assuming  that  the  results  from  CoHOST  can  be  believed,  therefore,  then  the  focus  on 
improving  the  ability  of  the  battalion  commander  to  perform  all  the  required  duties  for  the  job 
would  be  to  improve  the  tools  available  to  maintain  an  assessment  of  the  surrounding  situation 
along  with  longer  range  and  improved  conununications  systems  to  enable  this  individual  to 
always  be  in  communication  with  all  the  members  of  the  team. 

5.5.  Major  Findings. 

The  significance  of  the  results  of  this  research  is  primarily  in  the  area  of  the  development 
of  new  methods  for  the  use  of  pre-existing  computer  simulations  of  human  performance  to  allow 
optimized  evaluations  of  their  predictive  performance  abilities.  Regardless  of  the  original 
predictive  algorithmic  frinctions  that  might  exist  in  the  simulation  and  the  questions  that 
originally  drove  the  development  of  these  algorithms,  the  use  of  experimental  design  and 
regression  techniques  can  allow  these  functions  to  assume  a  new  role  in  predictions  of  how  to 
optimize  those  frmctions  to  maximize  human  performance.  A  series  of  guidelines,  derived  from 
these  methods,  are  illustrated  in  the  specific  case  of  the  work  domain  described  by  the  CoHOST 
simulation  to  comment  on  how  to  improve  the  performance  of  command  and  control  teams  in  an 
Army  TOC. 

The  guidelines  presented  here  fall  into  three  categories.  These  categories  are 
experimental  design,  simulation,  and  analysis.  See  Table  26  for  a  summary  of  the  guidelines  and 
sections  5.5.1  and  5.5.2  for  complete  descriptions. 
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Table  26  -  Guidelines  For  Simiilation  Based  Experimentation 


Type 

# 

Description 

Experimental  Design 

1 

Determine  level  of  design. 

Experimental  Design 

D 

Determine  lowest  number  of  treatments  required  for  desired  resolution. 

Experimental  Design 

3 

Determine  of  the  design  needs  to  be  augmented. 

Experimental  Design 

4 

Use  manual  calculations  as  required  to  extend  the  analysis. 

Experimental  Design 

5 

Consider  the  use  of  unbalanced  treatments  to  reduce  required  number  of  simulation  runs. 

Simulation 

1 

Establish  an  acceptable  level  of  risk  for  use  of  the  simulation. 

Simulation 

2 

Determine  the  number  of  required  replications  of  the  simulation. 

Simulation 

3 

Use  dummy  IV  to  have  simulation  self  report  its  significance  levels. 

Analysis 

1 

Determine  whether  the  data  should  be  standardized. 

Analysis 

2 

Interpret  analysis  results  in  terms  of  real  world  constructs. 

5.5.1.  Methodology  for  Human  Performance  Analysis  Using  Experimentally  Interrogated 

Simulations. 

The  first  step  in  experimentally  interrogating  a  simulation  is  to  develop  an  appropriate 
experimental  design.  This  generally  is  an  iterative  process  that  starts  first  with  a  basic  design 
that  is  then  modified  according  to  operational  constraints  and  requirements.  This  methodology 
can  be  dynamic  in  that  the  modifications  conform  to  the  specifications  of  the  simulation  used, 
however,  many  of  the  steps  and  considerations  can  be  generalized  to  the  point  that  they  are  not 
unique  to  a  specific  simulation.  With  the  example  simulation  as  provided  by  CoHOST,  after 
identifying  the  number  of  independent  measures  and  treatment  levels,  a  basic  2^  experimental 
design  emerged.  The  first  step  or  guideline  is  to  ask  the  question, 

>  Experimental  Design  Guideline  (1).  Can  the  design  be  conducted  as  a  full 
factorial  experiment?  If  the  answer  is  “yes”  then  conduct  it.  If  the  answer  is  “no” 
then  precede  to  guideline  2. 

For  CoHOST  this  means  32  treatment  combinations  resulting  from  the  2^  design.  First, 
there  was  not  enough  time  to  make  all  the  simulation  runs  to  support  these  treatments,  and 
second,  a  full  factorial  experiment  would  be  very  inefficient  as  it  would  collect  data  to  support 
higher  level  interactions  that  could  not  be  interpreted.  For  situations  such  as  this  the  next 
guideline  is  invoked; 

>  Experimental  Design  Guideline  (2).  Reduce  the  design  fractionally  until  the 
lowest  number  of  treatments  that  will  satisfy  the  resolution  requirement  is 
provided. 
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For  CoHOST  this  involved  evaluating  replicate  and  ‘/4  replicate  fractional  factorial 
designs.  A  Vi  replicate  design  of  16  treatments  would  provide  for  a  Resolution  V  design  that 
resolves  all  the  main  effects  and  two  way  interactions.  Since  only  the  main  effects  are  of  interest 
^  and  the  two  way  interactions  cannot  be  interpreted  for  CoHOST,  this  resolution  is  still  too 

inefficient  and  subject  to  further  reduction.  A  V4  replicate  design  of  8  treatments  provides  for 
Resolution  HI  that  will  resolve  all  the  main  effects  and  some  of  the  two-way  interactions, 
i  Proceeding  on  to  a  1/8  design  would  not  resolve  the  main  effects  and  so  a  replicate  fractional 

factorial  design  is  considered  the  optimiun  level  for  CoHOST.  The  next  guideline  looks  at  the 
level  of  analysis  capable  from  the  design: 

>  Experimental  Design  Guideline  (3).  Can  higher  order  effects  from  the  main 
effects  and  /  or  interactions  be  examined  with  this  design?  If  “yes”  then  proceed 
with  the  design,  if  “no”  then  investigate  whether  the  design  can  be  augmented 
with  additional  treatments. 

For  the  ‘/i  replicate  2®  fractional  factorial  design  in  CoHOST  this  answer  is  “no”.  While 
only  two  treatment  levels  provides  for  efficient  execution  of  the  simulation,  it  limits  the  resulting 
analysis  to  only  an  evaluation  of  the  linear  components  of  the  main  effects  and  does  not  support 
a  response  surface  or  dependent  variable  response  analysis  of  higher  order  components  for  these 
variables.  The  CoHOST  data  was  then  examined  for  possible  additional  treatment  combinations. 
As  the  existing  two  treatments  were  generated  from  plus  and  minus  deltas  from  the  original  data 
in  the  model,  this  existing  data  could  be  considered  as  a  center  point  treatment  as  is  normally  the 
case  in  a  central  composite  design.  Using  this  treatment  now  provided  a  third  treatment  level  for 
the  design  with  the  valued  added  capability  to  theoretically  evaluate  higher  order  components  of 
the  output  data.  The  cost  of  adding  this  third  treatment  level  to  a  2^  design  is  that  it  is  no  longer 
^  a  true  2^  design.  Also,  unless  the  treatments  are  recalculated  for  a  3  level  design  it  remains  a  2 

level  design  that  is  augmented  that  provides  some  capability  for  examining  higher  order  effects 
but  does  not  generate  enough  data  points  to  support  a  full  calculated  quadratic  evaluation  of  the 
*  output.  In  order  to  keep  the  number  of  simulation  runs  within  a  manageable  level  with  CoHOST 

the  decision  was  made  to  stay  with  the  2  level  design  and  use  manual  workaroimd  methods  to 
evaluate  the  higher  order  effects  as  closely  as  possible.  This  leads  to  the  next  guideline  which 
addresses  how  to  address  data  that  supports  analysis  higher  than  the  original  design  but  does  not 
generate  enough  data  for  a  full  higher  level  analysis. 
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>  Experimental  Design  Guideline  (4).  If  full  analysis  of  the  output  data  is  not 
possible  to  the  desired  level,  use  manual  calculations  and  procedures  from  the 
existing  data  to  make  predictions  of  what  the  higher  order  interactions  might  be. 

For  CoHOST  the  desire  was  to  investigate  whether  or  not  there  were  quadratic 
components  of  the  main  effects.  The  third  center  point  treatment  level  provided  this  ability,  but 
as  the  design  remained  basically  a  two  treatment  level  design  with  not  enough  data  collected  to 
support  a  full  three  level  design  a  direct  statistical  calculation  of  the  qiiadratic  effects  was  not 
possible.  The  manual  workarounds  involved  the  calculation  of  the  response  surface  maximum 
values  for  each  DV/TV  combination  and  plotting  the  DV  response  curves  for  each  3  value 
dataset.  Subjective  interpretation  of  these  curves  supported  by  LOF  calculations  that  predicted 
the  presence  of  higher  order  effects  was  then  used  to  make  the  predictions  as  to  the  possibility  of 
the  presence  of  the  higher  order  effects. 

If  the  number  of  required  simulation  runs  to  support  all  the  treatments  in  the  design  must 
be  reduced  further  to  achieve  the  ability  to  run  the  experiment  then  unbalanced  treatments  can  be 
considered  according  to  the  next  guideline: 

>  Experimental  Design  Guideline  (5).  To  minimize  the  required  number  of 
computer  runs,  unbalance  the  treatments  by  reducing  the  Actional  factorial 
treatments  to  one  and  repeat  the  center  point  treatment  as  many  times  as  necessary 
to  achieve  an  accq)table  error  level. 

This  was  performed  with  CoHOST  and  the  center  point  treatment  was  repeated  1 1  times. 
This  resulted  in  a  total  of  (8  fractional  factorial  +11  center  point  = )  19  computer  runs  which  is 
significantly  less  than  the  required  27  runs  if  each  treatment  had  been  repeated  3  times 
simulating  3  subjects  per  treatment.  It  is  speculated  that  the  negative  aspect  of  this  unbalancing 
of  treatments  with  only  one  repetition  of  the  fractional  factorial  treatments  is  that  random 
variability  in  the  model  was  higher  than  would  have  been  the  case  with  higher  number  of 
repetitions  of  each  treatment.  Additional  computer  runs  would  be  required  for  a  full  3  repetition 
(or  more)  per  treatment  case  of  complete  balancing  to  verify  this  speculation. 

5.5.2.  Model  Requirements  for  Analysis  Using  Experimentally  Interrogated  Simulations. 

The  literature  is  replete  with  descriptions  of  requirements  for  conducting  studies  using 
simulation  (Banks,  1998;  Banks  et  al.,  1996;  Knepell  and  Arangno,  1993;  Laughery  and  Corker, 
1992;  Law,  1997;  Law  and  Kelton,  1991;  Law  and  Kelton,  2000;  Pew  and  Mavor,  1997; 
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Zachary,  Ryder,  Hicinbothom,  and  Bracken,  1997),  however,  based  on  the  observations  gathered 
here,  a  few  points  should  be  emphasized.  First,  and  foremost,  the  simulation  must  be  trusted. 

>  Simulation  Guideline  (1).  The  simulation  must  have  been  evaluated  to  the  point 
that  an  acceptable  level  of  risk  is  established  whereby  the  users  are  willing  to  risk 
using  the  simulation  vice  live  testing  thereby  enabling  the  simulation  to  be 
considered  a  ‘black  box’  that  describes  a  system  to  an  acceptable  level  of 
resolution  and  used  to  answer  questions  about  that  operational  system. 

The  original  development  of  the  CoHOST  simulation  included  an  extensive  verification  and 
validation  process  followed  by  a  formal  failrure  modes  analysis  to  identify  its  weaknesses.  This 
was  followed  by  a  risk  assessment  performed  by  the  model  developers  that  satisfied  the  users  of 
the  simulation  at  that  time.  Although  the  project  ended  before  the  simulation  received  full 
accreditation,  it  satisfied  all  of  the  user  requirements  then  placed  upon  it  for  use. 

After  the  determination  of  the  type  and  extent  of  the  experimental  design  to  be  placed 
upon  it,  the  simulation  needs  to  be  evaluated  for  random  variability. 

>  Simulation  Guideline  (2).  The  number  of  replications  to  be  executed  per 
simulation  treatment  run  needs  to  be  determined  to  account  for  random  variability 
in  the  model. 

CoHOST  was  evaluated  according  to  a  method  in  the  literature  (Banks  et  al.,  1996)  and 
was  found  to  require  15  replications  to  satisfy  this  requirement.  This  simulation  is  typical  of 
most  simulations  in  that  it  invokes  multiple  dependent  measures  and  requires  that  each  of  them 
be  evaluated  by  the  process.  The  dependent  variable  that  resxilts  in  the  most  required  replications 
then  becomes  the  defining  variable  for  this  requirement.  After  all  the  replications  are  executed 
for  a  simulation  run  the  data  from  all  the  replications  is  averaged  to  produce  the  data  for  the 
treatment  condition  of  the  simulation  run. 

While  the  above  guidelines  merely  restate  accepted  practices  in  a  form  applicable  to  this 
type  of  study,  the  next  guideline  is  not  known  to  exist  in  the  literature  and  overcomes  a  problem 
common  in  regression  equation  analyses.  While  attempting  to  fit  regression  equations  to  the  data 
and  determine  which  elements  are  significant  and  which  should  be  eliminated,  the  researcher  is 
commonly  left  to  his  or  her  own  devices  to  rationalize  a  level  of  significance  to  be  used  to  accept 
or  reject  a  particular  data  parameter.  To  overcome  this  subjective  requirement  computer 
simulations  can  be  queried  to  reveal  what  the  level  of  significance  for  random  variability  in  the 
simulation  is  for  the  given  set  of  input  characteristics. 
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>  Simulation  Guideline  (3).  Use  a  dummy  independent  variable  in  the  experimental 
design  for  the  simulation  and  then  make  computer  runs  that  supposedly  test  this 
variable.  From  the  resulting  statistical  analysis  use  the  p  value  for  this  value  as 
the  cutoff  threshold  to  determine  significance  for  the  other  variables. 

This  process  allows  for  the  exact  determination  of  unmanipulated  variability  in  the 
simulation  which  can  be  used  in  place  of  the  traditional  table  lookup  values  of  .05,  .01,  or  .001 
and  regression  techniques  such  as  forward,  backward,  and  stepwise  selection  of  regression 
equation  terms.  This  technique  was  used  in  the  CoHOST  model  and  was  compared  with  die 
results  from  the  standard  forward,  backward,  and  stepwise  regression  selections  using  a 
statistical  analysis  package.  The  resulting  equations  caused  more  terms  in  the  regression 
equation  to  be  dropped  thereby  indicating  a  closer  fit  of  the  final  equations. 

Although  not  explicitly  a  simulation  method,  considerations  must  be  given  to  the  nature 
of  the  data  produced  by  it  and  how  that  data  is  to  be  evaluated  in  the  analysis  phase.  Because  of 
the  ability  of  computer  simulations  to  generate  large  quantities  of  data  very  quickly,  this 
consideration  can  take  on  added  importance  to  preclude  having  a  large  quantity  of  data  after  the 
simulation  runs  that  might  be  overly  difficult  or  impossible  to  resolve  in  the  manner  desired. 
Although  computer  simulation  can  be  fast,  this  speed  is  often  used  to  generate  more  output  data 
than  would  be  normally  possible  and  the  time  may  not  exist  to  rerun  the  computers  if  the  data  is 
not  in  a  usable  form.  In  human  factors  research  one  of  the  considerations  for  the  output  data  is 
whether  the  dependent  measures  will  be  evaluated  individually  or  as  a  group.  If  each  dependent 
variable  is  to  be  evaluated  independently  then  the  output  data  might  be  xised  as  produced.  If  the 
dependent  measures  are  to  be  considered  as  a  group  then  other  procedures  may  be  performed 
first. 

>  Analysis  Guideline  (1).  If  multiple  dependent  measures  are  to  be  analyzed  as  a 
group  for  significance  effects,  the  data  should  be  standardized  before  &e  analysis 
is  conducted. 

The  dependent  measures  in  the  CoHOST  simulation  could  not  be  directly  compared  to 
each  other  because  of  the  values  involved.  For  example,  percent  utilization  was  on  a  100% 
scale,  taskload  was  a  dimensionless  quantity,  and  the  task  based  measures  were  all  counts.  By 
standardizing  these  measures  to  a  mean  of  “0”  and  a  standard  deviation  of  “1”  the  data  all 
became  normalized  along  a  common  scale  and  diey  could  all  be  compared  together. 
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Analytical  procedures  and  statistical  methods  can  provide  a  good  description  of  what  the 
data  is  predicting,  however,  the  descriptions  of  such  results  many  times  can  only  be  understood 
by  those  conversant  in  statistical  and  research  procedures.  It  is  to  the  system  developers,  who 
originally  posed  the  questions  that  research  and  simulations  try  to  answer,  that  the  results  need  to 
be  tailored.  Individual,  team,  and  macroergonomic  performance  descriptions  and  models  can  be 
iised  to  satisfy  this  need. 

>  Analysis  Guideline  (2).  The  final  data  and  study  conclusions  should  be 
interpreted  /  translated  into  real  world  constructs  and  meaningful  descriptions 
pertinent  to  the  original  system  domain  that  generated  the  research. 

This  study  pursued  this  goal  by  taking  the  results  from  the  CoHOST  interrogation  and 
relating  it  to  a  human  performance  model  from  the  team  performance  and  macroergonomic 
literature.  Further  descriptions  and  examples  using  a  scenario  based  approach  have  attempted  to 
illustrate  how  the  realizations  of  the  conclusions  from  the  study  can  have  meaning  in  the 
originating  work  domain. 

5.5.3,  A  Simulation  Example  -  U.S.  Army  Battalion  Command  And  Control  Team. 

The  procedures  were  developed  in  this  thesis  according  to  the  structure  and  requirements 
of  the  CoHOST  model.  While  they  are  therefore  constrained  by  the  capabilities,  limitations,  and 
organization  of  CoHOST,  they  nevertheless,  embody  constructs  and  concepts  that  can  apply  to  a 
wide  range  of  scenarios  and  simulations.  The  requirement  to  evaluate  real  world  scenarios  that 
may  not  yet  exist,  may  be  too  costly  to  operate,  may  be  too  dangerous  to  the  operator  for  full 
scale  testing  or  may  just  take  too  long  to  evaluate  can  be  readily  addressed  in  the  rapidly 
evolving  world  of  computer  simulation.  Many  research  procedures  that  are  applicable  to 
conventional  empirical  research  are  equally  applicable  in  the  computer  simulation,  however,  the 
unique  capabilities  that  simulation  based  research  brings  to  the  researcher  can  often  be  enhanced 
by  procedures  that  are  unique  to  simulation  based  research  or  that  are  not  even  a  consideration  in 
conventional  research. 

Use  of  the  simulation  requires  answers  to  questions  related  to  acceptance  of  risk,  how 
much  is  good  enough  in  the  replication  provided  by  the  simulation  and  a  very  important  role  for 
subject  matter  experts  to  define  what  the  simulation  needs  to  replicate  and  then  judge  if  the 
replication  is  good  enough. 
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The  CoHOST  computer  simulation  addresses  work  domain  issues  that  the  Army  is  still 
trying  to  resolve  today.  While  it  is  a  very  Army  specific  investigation  into  human  computer 
interface  (HCI)  issues,  the  basic  nature  of  the  work  domain  and  the  procedures,  concepts  and 
^  constructs  used  in  the  simulation  are  not  limited  to  just  the  Army  or  military  applications.  Any 

work  team  that  fimctions  in  a  decision  oriented,  high  stress,  time  oriented  work  situation  could 
be  addressed  by  this  type  of  simulation  which  could  provide  insights  as  to  what  work  activities, 

4  priorities,  and  coordination  activities  are  most  important  to  the  successful  functioning  of  the 

team. 

To  apply  a  CoHOST  simulation  in  another  work  domain  that  could  capitalize  on  the 
methodologies  contained  in  this  thesis  would  require  primarily  database  type  descriptions  of  that 
work  domain.  The  job  tasks  would  have  to  be  identified  and  applied  to  the  human  performance 
taxonomy.  SME’s  fi'om  the  new  domain  would  need  to  identify  which  taxons  apply  to  each  task 
performed  by  individuals  in  the  work  domain  and  the  level  of  the  application.  Work  and 
information  flow  activities  would  need  to  be  diagrammed  and  charted  and  individual  task  based 
activities  associated  with  each  construct  in  the  work  flow  charts.  From  these  data  the 
MicroSaint™  computer  code  would  be  revised  to  reflect  the  new  activities  in  the  new  domain. 

For  the  case  of  the  CoHOST  model,  the  specifics  that  this  model  and  set  of  research 
guidelines  and  methodologies  establish  are  all  areas  for  which  there  is  no  known  correct  answer 
as  it  applies  to  work  domain  situations  that  are  in  near  constant  development  along  almost 
iterative  development  lines.  While  this  model  was  programmed  to  investigate  command  and 
control  at  the  battalion  level  because  that  is  what  the  original  clients  for  the  effort  were  interested 
in,  the  activities  and  interactions  of  this  23  member  work  team  are  very  similar  to  span  of  control 
interactions  for  commanders  at  ever  higher  levels  of  the  chain  of  command.  The  battalion 
^  commander  conunands  a  1 000  member  battalion,  the  brigade  commander  a  5,000  member 

brigade,  the  division  commander  a  15,000  member  division  and  so  on,  the  critical  members 
^  working  directly  with  the  commander  at  any  level  constitute  a  similar  work  team  at  each  of  the 

levels.  The  main  difference  is  that  team  members  at  higher  levels  have  access  to  more  resources 
as  the  level  goes  up,  but  the  basic  activities  they  perform  are  very  similar.  The  questions  that 
CoHOST  addresses  as  to  what  the  commiinication  requirements  are,  who  needs  to  make  what 
decision,  and  who  needs  to  take  what  action  for  each  situational  requirement  are  all  able  to  be 
investigated  to  determine  if  decision  making,  battlefield  assessment,  development  of  accurate 
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mental  models,  sharing  of  those  models,  evaluating  the  impact  of  decisions  or  inter  /  intra  team 
communications  are  the  most  important  and  required  activity  at  any  one  time.  Once  these 
requirements  and  their  priorities  are  quantified  then  steps  can  begin  to  tailor  the  work  situation  to 
give  priority  to  those  activities. 

5.6.  Topics  For  Future  Research. 

This  thesis  has  set  the  stage  for  continuing  research  in  several  areas.  The  work  covered 
in  this  study  is  only  the  beginning  of  what  is  envisioned  as  an  ongoing  effort  in  simulation  based 
experimental  research  with  a  focus  in  team  performance  especially  in  the  area  of  military 
command  and  control.  While  the  CoHOST  simulation  has  been  used  as  a  tool  to  develop  and 
refme  methodological  procedures  for  the  use  and  analysis  of  simulation  based  research,  the 
topical  area  of  this  simulation  provides  a  rich  medium  for  investigative  analysis  and  development 
of  procedures,  priorities,  and  mental  schema  as  it  applies  to  team  based  command  and  control. 
The  age  old  quandary  of  what  came  first,  ‘the  chicken  or  the  egg’,  applies  to  this  situation  where 
simulations  like  CoHOST  and  others  like  it  provide  the  medium  for  the  investigative  analysis, 
but  logically  defined  methodologies  and  procedures  for  the  use  of  the  model  and  the  conduct  of 
the  resulting  analysis  provide  the  ability  to  inteipret  and  understand  what  is  and  is  not  being 
predicted. 

5.6. 1 .  Implications  For  The  Methodology. 

The  implication  for  the  research  methods  described  here,  some  of  which  were  revealed 
during  the  pursuit  of  other  priorities,  is  that  continued  refinement  and  optimization  of  both  the 
data  gathering  and  analysis  process  should  be  pursued.  This  effort  needs  to  be  extended  into 
several  areas.  First,  the  compromise  between  the  desire  for  as  few  a  number  of  simulation  runs 
as  possible  and  the  desire  for  the  ability  to  fully  explore  the  output  data  statistically  needs  to  be 
better  resolved.  While  it  may  not  always  be  feasible  to  predict  how  much  resolution  should  exist 
in  the  data  until  the  process  of  analyzing  and  data  mining  begins,  careful  and  well  thought  out 
experimental  designs  can  allow  the  establishment  of  the  level  of  analysis  that  is  capable  before 
the  data  is  collected.  Each  time  the  simulation  is  run  in  response  to  a  predictive  question  the 
modifications  to  the  procedures  and  requirements  to  tailor  the  methodology  and  simulation  to  the 
circumstance  of  the  test  can  add  to  the  knowledge  base  of  the  methodology.  In  addition,  the 
analysis  in  this  thesis  only  involved  multiple  looks  at  univariate  data.  The  multiple  dependent 
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measures  produced  by  CoHOST  and  almost  any  other  comparable  model  are  ripe  for 
multivariate  examination  and  could  start  to  provide  the  ability  to  answer  what  some  of  the  effects 
from  second  order  interactions  might  be. 

^  Second,  future  research  can  focus  on  the  concept  of  extending  or  improving  the  way  the 

simulation  model  self-reports  its  limits.  The  use  of  dummy  variables  in  the  experimental  design 
needs  to  be  more  thoroughly  investigated.  The  issue  of  unmanipulated  variability  and  ‘random 
*  number  noise’  should  be  examined  with  the  intent  of  reducing  it  as  much  as  possible  while  still 

allowing  explicit  random  variation  to  simulate  the  effects  of  individual  human  variability  in 
performance.  Tests  with  different  random  variates  and  random  distributions  can  establish  which 
profile  best  meets  the  criteria.  This  could  also  be  compared  to  similar  studies  using  live  human 
test  subjects  (if  they  exist  and  /  or  can  be  conducted)  to  try  and  fit  simulated  random  variability 
as  close  as  possible  to  actual  human  variability. 

5.6.2.  Implications  For  The  CoHOST  Example. 

The  use  of  CoHOST  in  this  thesis  involved  its  use  in  an  unmodified  state  except  for  the 
performance  data  as  it  is  stored  inside  the  MicroSainf computer  simulation  file.  This  was 
important  to  illustrate  how  preexisting  simulations  can  be  utilized  for  applications  beyond  their 
original  intent.  However,  the  CoHOST  example  represents  a  defining  nature  of  the  research 
requirements  for  future  work.  Questions  about  the  Army  command  and  control  domain  and  how 
to  improve  it  continue  to  be  a  focus  of  Army  force  and  system  developers.  Therefore,  future  use 
of  computer  simulation  to  investigate  human  performance  requirements  in  the  military  command 
and  control  domain  need  to  focus  on  models  like  CoHOST  as  an  exploratory  tool  to  look  at  the 
demands  that  these  evolving  work  domains  might  or  do  place  on  human  operators.  To  be  sure, 

^1,  CoHOST  is  not  unique  in  this  application  area.  One  simulation  (Essens,  Post,  and  Rasker,  2000) 

takes  a  similar  but  separate  approach  where  the  computational  entities  in  the  model  are  organized 
around  “information  entities  and  the  functions  that  generate  or  transform  them”  while  CoHOST 
focuses  on  information  entities  and  their  processing  through  the  team  organizations.  Therefore, 
while  the  current  CoHOST  model  is  not  currently  in  continued  development,  there  are 
refinements  to  be  made  that  can  improve  its  ability  to  simulate  the  domain.  Indeed,  any 
simulation  of  the  real  world  is  a  continued  candidate  for  improvement  as  it  will  always  only  be 
an  approximation  of  the  actual  world  state  and  the  more  that  development  is  invested  in  it  the 
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tighter  the  approximation  will  become.  The  possible  improvement  areas  for  CoHOST  or  a 
CoHOST  derivative  or  next  generation  simulation  fall  into  two  primary  areas.  These  are  data 
structure  improvements  and  model  algorithmic  improvements. 

Data  structure  improvement  needs  in  CoHOST  include,  first,  a  revalidation  of  the 
taxonomy  used  to  quantify  the  performance  levels  of  individual  tasks.  While  Fleishman’s 
taxonomy  provides  a  general  descriptive  base  for  this  kind  of  application,  it  is  not  widely 
accepted  in  the  work  performance  community  and  other  descriptive  formats  such  as  Wickens’ 
dual  task  methodology  (Wickens  and  Hollands,  2000)  or  some  of  the  cognitive  and  multiple  task 
analysis  procedures  covered  by  work  of  researchers  like  Klein  and  Damos  (Damos,  1991 ;  Klein, 
2000)  might  provide  a  more  current  firamework.  Second,  the  SME  opinion  based  data  needs  to 
be  totally  revised  and  updated.  It  is  envisioned  that  simulation  based  human  performance 
research  needs  to  be  extensively  supported  by  expert  opinion  on  how  and  which  parts  of  the 
performance  taxonomy  apply  to  individual  task  elements.  The  more  SME’s  that  can  be  involved 
the  more  refined  the  data  becomes.  This  querying  of  SME  opinion  should  be  conducted  in 
parallel  with  naturalistic  observation  of  human  performance  for  each  of  the  operators  in  the 
simulation  for  each  of  the  task  based  conditions  in  the  simulation.  Where  live  empirical  research 
typically  gathers  performance  data  from  a  few  subjects  under  strictly  controlled  conditions  over  a 
short  period  of  time,  simulated  empirical  research  of  human  performance  should  pursue  these 
same  efforts  only  after  naturalistic  observations  of  live  human  performance  in  real  world 
situations  validates  the  structure  and  format  of  individual  task  performance  constructs  so  that 
they  can  be  combined  and  observed  in  controlled  environments  that  can  be  exactly  repeated 
under  varying  conditions  in  the  simulation. 

Future  research  for  simulations  like  CoHOST  itself  are  envisioned  as  a  next  generation 
effort  to  expand  the  simulation  firom  just  a  network  based  task  performance  emulator  and  tracker 
that  is  an  apt  description  of  the  basic  functionality  in  CoHOST,  to  a  whole  series  of  interlinked 
performance  modules  that  evaluate  incoming  information  as  to  parameters  such  as  its  type, 
priority,  impact,  and  potential  impact  that  then  produce  information  processing  results.  The 
previously  referred  to  naval  command  center  model  (Essens  et  al.,  2000)  describes  some  of  the 
components  that  this  kind  of  model  would  include: 

•  Organization  model. 

•  Agent  model. 
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•  Knowledge  model. 

•  Event  handling  model. 

•  Coordination  model. 

•  Means  model. 

•  Information  model. 

•  Function  information  model. 

A  fully  implemented  simulation  of  this  type  would  include  complete  performance  databases  for 
each  of  the  functional  areas.  Each  functional  model  would  be  an  independent  program  perhaps 
running  on  separate  computers  for  processor  efficiency  and  would  be  supported  by  its  own 
naturalistically  derived  and  developed  performance  data  for  that  functional  sub-area. 

5.7.  Conclusions. 

The  next  step  for  this  research  is  clear  and  encompasses  both  the  methodology  (i.e., 
process)  and  the  results  (or  product).  Continued  refinement  of  the  experimental  design  is  needed 
to  support  a  more  extensive  analysis  capability  while  still  keeping  the  computing  requirements  to 
a  minimum. 

The  product  of  this  research  is  the  predictive  results  that  can  be  obtained  from  models 
like  CoHOST  using  the  methodologies  developed  to  support  and  drive  it.  For  this  case,  referring 
to  the  original  project  that  developed  the  CoHOST  models,  it  is  noted  that  one  of  the  previous 
conclusions  was  that  increased  efficiency  of  communications  systems  as  they  were  being 
designed  actually  degraded  the  cognitive  performance  of  the  decision  makers.  Of  course,  the 
primary  decision  maker  for  this  work  group  is  the  battalion  commander.  The  results  from  this 
thesis  correspond  with  these  observations  with  the  conclusion  that  the  battalion  commander’s 
primary  attentional  focus  is  based  on  developing  and  maintaining  a  mental  model  that  allows  a 
continuing  understanding  of  the  situational  picture  associated  with  the  spread  of  activities  in  the 
battlefield  being  observed  while  trying  to  direct.  Both  of  these  observations,  arrived  at  from 
totally  different  approaches  to  the  use  of  the  CoHOST  model,  very  directly  point  to  changes  in 
the  work  environment  for  the  battalion  commander  that  could  be  made  to  optimize  the  required 
cognitive  capabilities.  The  original  study  illustrated  that  changes  should  be  made  to  increase  the 
amount  of  time  available  for  the  proactive  think  ahead  reasoning  type  of  task  performance.  This 
thesis  focuses  this  further  by  illustrating  that  the  tasks  of  monitoring  and  understanding  the 
situational  picture  is  the  most  important.  Effective  performance  in  these  activities  allow  the 
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battalion  commander  the  ability  to  operate  more  effectively  in  a  leadership  role  in  the  team  and 
directly  places  this  individual  within  activities  as  described  by  models  such  as  the  TEM. 
Therefore,  automated  aids  developed  for  this  operator  should  not  focus  on  just  providing  more 
information  arriving  at  quicker  rates  that  requires  more  attention  to  decipher  and  understand. 
Rather,  the  work  environment  should  be  optimized  to  provide  displays  that  show  battlefield 
intelligence  information  that  is  fused  to  indicate  its  meaning.  For  example,  an  automated  display 
should  not  show  information  such  as  ‘three  BTR  60  armored  vehicles  are  moving  into  the 
sector’.  Instead  it  should  reflect  the  information  that  scout  elements  preceding  an  enemy 
mechanized  infantry  division  are  approaching  and  this  indicates  that  there  is  one  hour  before 
being  faced  with  a  massive  attack  with  a  probable  force  ratio  of  9  to  1  against  the  fnendly 
positions.  Now,  instead  of  spending  time  trying  to  figure  out  what  the  meaning,  if  any,  of  the 
presence  of  these  3  vehicles  is,  the  battalion  commander  can  spend  the  precious  hour  available 
by  directing  and  coordinating  a  response  to  the  impending  attack. 

The  research  methods  described  by  this  thesis  consist  primarily  of  refinements  of 
conventional  approaches  that  would  and  do  apply  in  general  to  empirical  research.  The  tailoring 
of  the  techniques  for  implementation  within  a  simulation  based  investigative  environment  along 
with  procedures  for  operation  of  the  simulation  itself  is  considered  the  primary  contribution  of 
this  thesis  to  the  knowledge  base.  However,  it  is  the  application  of  these  techniques  using  an 
appropriate  tool  such  as  CoHOST  that  enables  answers  and  predictive  evaluations  to  be  made 
about  current  system  worlds  that  are  too  difficult  to  investigate  by  conventional  means  or  for 
future  system  worlds  that  do  not  yet  exist. 
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Appendix  A  -  Determination  of  %  Replicate  Fractional  Factorial  Treatment  Combinations. 
1.  List  all  possible  treatment  combinations  for  a  2^  full  factorial  design: 
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3.  Sub-divide  into  4  blocks  according  to  the  C2  identity  relationship: 

Second  Identity  Relationship:  C2:  XI  +  X3  +  X4  =  0  (Mod.  2)  *  ACE  Steps: 

1.  Evaluate  Factors  A, C,E  for  Mod.  2, 


2.  Sort  on  the  Mod.  2 

evaluation  to  identify  the  2  blocks 
for  this  identity  relationship. 
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Comparing  this  selection  to  the  SAS  ADX  Experimental  Design  module,  this  agrees  with  the 
treatment  block  selected  by  SAS  as: 


* 
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Appendix  B  -  KSA  Treatment  Condition  1  Setup  Table. 


Appendix  B:  KSA  Data  Configuration  for  Treatment  Condition  #1 :  PMMMM 
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28-Send  Digital  Information 

0.00 

1.20 

0.00 

15 

Communicate  and  Report 

0020 

WRlffEN  COMPRiHENSl^ 

01 -Receive  and  Record/Analog 

1.54 

1.20 

1.85 

16 

Communicate  and  Report 

0020 

WRlffEN  COMPREH^NSI^ 

02-Pass  information 

0.00 

1.20 

0.00 

17 

Communicate  and  Report 

0020 

IWRITTEN  COMPREHENSK 

03-Listen-Receive  Information 

1.54 

1.20 

1.85 

18 

Communicate  and  Report 

0020 

WRITTEN  COMPREHENSk 

04-Secondary  Monitor 

1.54 

1.20 

1.85 

19 

Communicate  and  Report 

0020 

WRITTEN  COMPREHENSK 

05-Log  Message 

1.54 

1.20 

1.85 

20 

Communicate  and  Report 

0020 

WRITTEN  COMPREHENSI^  06-Route  (Outside  the  Section) 

0.00 

1.20 

0.00 

21 

Communicate  and  Report 

0020  jWRITTEN  COMPREHENSK 

07-Send  Message 

0.00 

1.20 

0.00 

22 

Communicate  and  Report 

0020 

WRITTEN  COMPREHENSK 

08-Verbal  Order 

0.00 

1.20 

0.00 

23 

Communicate  and  Report 

0020 

WRITTEN  COMPREHENSK 

09-Roll  Up  Reports 

1.54 

1.20 

1.85 

24 

Communicate  and  Report 

0020 

*  WRITTEN  COMPREHENSK 

10-Call  to  Conference 

0.00 

1.20 

0.00 

25 

Communicate  and  Report 

0020 

WRITTEN  COMPREHENSK 

26-Receive  Digital  Message 

1.54 

1.20 

1.85 

26 

Communicate  and  Report 

0020 

IWRITTEN  COMPREHENSK 

27-Input  data  Into  Computer 

1.54 

1.20 

1.85 

27 

Communicate  and  Report 

0020 

, WRITTEN  COMPREHENSK 

28-Send  Digital  Information 

Tm . i 

1.20  1 

1,85 

28 

Communicate  and  Report  I 

0030 

joi^L  ^rressTon 

01 -Receive  and  Record/Analog 

""2W\ 

1.20 

2.76 

29 

Communicate  and  Report  | 

0030 

lORAL  EXPRESSION 

02-Pass  Information 

2.36j 

r 

1.20 

2.76 

30 

Communicate  and  Report  i 

0030 

lORAL  EXPREi^lON 

03-Listen-Receive  Information 

0.00  ! 

r 

1.20 

0.00 

31 

Communicate  and  Report  i 

0030 

ORAL  EXPRESSION 

04-Secondary  Monitor 

i  0.00  i 

1.20 

0.00 

32 

Communicate  and  Report  ; 

0030 

[oral  expression 

05-Log  Message 

i  0.00  1 

1.20 

0.00 

33 

C^munic^  arid  ; 

0030  jORAL  EXPRESSION 

06-Route  (Outside  the  Section) 

1 aoo ! 

_ 

1.20 

0.00 

34 

Communic^^^^^ 

0030 

lORAL  EXPRESSION 

07’Send  Message 

I'Tso'i 

L 

1.20 

2.76 

35 

Communto^ 

0030 

lORAL  EXPRESSION 

OS-Verbai  Order 

\2M . 1 

i 

1.20 

2.76 

36 

Communicate  arid  Report 

0030 

lORAL  EXPRESSION 

09-Roli  Up  Reports 

'  6.00 ; 

! — 

1.20 

0.00 

37 

C^omriiunicate  j 

0030  lORAL  EXPRESSION 

10-Cail  to  Conference 

i  2.30 

lj 

1.20 

2.76 

38 

Cornmunicate  arid  Report  i 

0030 

lORAL  EXPRESSION 

26-Receive  Digital  Message 

i  0.00  ; 

i 

1.20 

0.00 

39 

Commu  riic^^^^^^^^  R®P9(1 .  J 

0030  jORAL  EXPRESSION 

27-Input  data  Into  Computer 

i'o.66 

1.20 

0.00 

40 

Communicate  and  Report  I 

0030 

jORAL  EXPRESSION 

28-Send  Digital  Information 

1  0.00 

1.20 

0.00 

mm 

Communicate  and  Report 

0040 

IwRifTEN  EXPRESSION 

01 -Receive  and  Record/Analog 

'  1.26 

1.20 

Uil 

\J2 

Communicate  and  Report 

0040 

IWRITTEN  EXPRESSION 

02-Pass  Information 

f  0.00 

1.20 

mm 

■a 

Communicate  and  Report 

0040  IWRITTEN  EXPRESSION 

03-Listen-Receive  Information 

1 

1.20 

0.00 

1  44 

Communicate  and  Report 

0040 

IWRITTEN  EXPRESSION 

04-Secondary  Monitor 

[“lie". 

1.20 

1.51 

mm 

Communicate  and  Report 

0040 

IWRITTEN  EXPRESSION 

05-Log  Message 

I  1.26 

1.20 

1.51 

mm 

Communicate  and  Report 

0040 

IWRITTEN  EXPRESSION 

06-Route  (Outside  the  Section) 

i  a66“ 

1.20 

0.00 

mm 

Communicate  and  Report 

0040 

IWRITTEN  EXPRESSION 

07-Send  Message 

1 aoo 

r~ 

1.20 

0.00 

mm 

Communicate  and  Report 

0040 

IWRITTEN  EXPRESSION 

08-Verbai  Order 

1  0.66 

1.20 

0.00 

mm 

Communicate  and  Report 

0040 

[WRITTEN  EXPRESSION 

09-RoIi  Up  Reports 

1  126 

r 

1.20 

■»HM 

Comrpuni(^te  and  ^ 

0040 

IWRITTEN  ^PRESSION 

10-Cail  to  Conference 

i  0.00 

1 

1.20 

0.00 

mm 

Comrnuriic^te 

0040 

IWRITTEN  EXPRESSION 

j  26-Receive  Digital  Message 

1  0.66 

1.20 

0.00 

Communicate  and  Report 

0040 

IWRITTEN  EXPRESSION 

27-input  data  into  Computer 

1.26 

1.20 

53 

Cqmmunic^^t^^ 

0040 

28-Sencl  Digital  information 

j  1.26 

L. 

1.20 

■Eil 

54 

Communicate  and  Report 

0050 

iMEMORIZATION 

1 01 -Receive  and  Record/Analog 

1  0.00 

3 

1.20 
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B 

c 

D 

E 

1 

S 

Communicate  and  Report 

005gj 

MEMORIZATION 

02-Pass  Information  1 

56 

Communicate  and  Report 

0050 

MEMORIZATION 

03-Llsten-Receive  Information  | 

57 

Communicate  and  Report 

0050 

MEMORIZATION 

04-Secondary  Monitor  ! 

58 

Communicate  and  Report 

0050 

MEMORIZATION 

05-Log  Message  | 

59 

Communicate  and  Report 

0050 

MEMORIZATION 

06-Route  (Outside  the  Section)  j 

El 

Communicate  and  Report 

0050 

MEMORIZATION 

07-Send  Message  j 

61 

Communicate  and  Report 

0050 

MEMORIZATION 

08-Verbal  Order  1 

62 

Communicate  and  Report 

0050 

MEMORIZATION 

69-Roli  Up  Reports 

63 

Communicate  and  Report 

0050 

MEMORl^flON 

1 6-Call  to  Conference  i 

64 

Communicate  and  Report 

0050 

MEMORIZATION 

26-Receive  Digital  Message  j 

Communicate  and  Report 

0050 

MEMORIZATION 

27-Input  data  Into  Computer 

Communicate  and  Report 

0050 

1  MEMORIZATION 

28-Senci  Digital  Information  | 

67 

Communicate  and  Report 

0060 

PROBLEM  SENSif  MTY 

01 -Receive  and  Record/Analog 

68 

Communicate  and  Report 

0060 

PROBLEM  SENSITIVITY 

02-Pass  information 

69 

Communicate  and  Report 

0066 

^PF^OBLEMSENSlfiviTY 

03-Listen-Receive  Information 

70 

Communicate  and  Report 

0060 

PROBLEM  SENSITIVITY 

04-Secondary  Monitor  1 

El 

Communicate  and  Report 

ooeo 

PFWDBLEMSENSifivifY 

05-Log  Message 

[72 

Communicate  and  Report 

0060 

PROBLEM  SENSITIVITY 

06-Route  (Outside  the  Section) 

mm 

Communicate  and  Report 

0060 

PROBLEM  SENSITIVITY 

07-Send  Message 

mm 

Communicate  and  Report 

ooeo 

PF^BLEM  SENSITIVITY 

08-Verbal  Order  j 

ra 

Communicate  and  Report 

0060 

PROBLEM  SENSITIVITY 

09-Roll  Up  Reports 

Communicate  and  Report 

0060 

PROBLEM  SENSITIVITY 

10-Call  to  Conference 

mm 

Communicate  and  Report 

0060 

PROBLEM  SENSITIVITY 

26-Receive  Digital  Message 

78 

Communicate  and  Report 

ooeo 

PROBLEM  SENSITIVITY 

27-Input  data  Into  Computer 

79 

Communicate  and  Report 

0060 

PROBLEM  SENSifivifY 

28-Send  Digital  Information 

El 

Communicate  and  Report 

~‘o676 

lOFdGIN^IlfY 

61 -Receive  and  Record/Analog 

81 

Cornmunicate  and  Report 

“  6670 

iORIGINALITY 

02-Pass  Information  j 

82 

"66^ ^ 

jORIGINALlfY'  '' 

03;Listen-Receiye  1  nfprmation  j 

Ka 

. 0070  ] 

lORlGiNALITY 

M-Secon^^  j 

El 

.J 

“66^ i 

IORIGINALITY 

Ea 

Co  mrn  u  n  i  cate  and  Report  i 

6676 . 

[originality 

Oe-Route  (Outside  the  Section)  | 

1  86 

l^mmunic^  and 

66^ 

iORIGINALlfY 

07-Send  Message  | 

Communicate  and  Report 

IORIGINALITY  ‘ 

OS-Verbal  Order  . . . . j 

Cornmunicate  and  ^ 

0070 

iOWGlNALITY' 

. . . . . . . J, 

Communicate  and  Report  i 

[6070 

joRIGlNAUfY . . 

l6-Ca^l[^to^^  j 

MtlM 

Communicate  and  Report  J 

r6676 

lORIGINAUTY . 

26-Receiye  D^  1 

El 

Communi^^^  ! 

1  66^ . 

rORIGfNALlfY . 

1  nto  Compute  \ 

M7M 

Communicate  and  Report  i 

foOTO . ^ 

iORIGiNALITY 

28-Send  Digital  Information  \ 

1  93 

Communicate  and  Report  1 

f  ooso  '  i 

IfluIncy  ofTdeas 

61 -Receive  and  Record/Analog  | 

El 

Communicate  and  Report 

1  0080 

[FLUENCY  OF  IDEAS 

62-Pass  Information  j 

1  95 

Communicate  and  Report  ; 

1  0080 

FLUENCY  OF  IDEAS 

63-Listen-Receive  Information  | 

pa 

Communicate  and  Report  i 

16680 

IFLUENCY  of  IDEAS 

64-Secondary  Monitor  \ 

pa 

Commu  ni(^^  J 

[fluency  OF  IDEAS 

05-Log  Message  | 

1  98 

Communicate  and  Report  i 

[ooeo “ 

[fluency  of  ideas 

Oe-Route  (Outside  the  Section)  \ 

El 

...i 

1  0080 

IFLUENCY  OF  IDEAS 

07-Send  Message  [ 

100 

Communicate  and  Report  { 

(  6066 

Fluency  of  ideas 

OS-Verbal  Order  | 

101 

Communicate  and  Report  1 

i "6680 

[FLUENCY  OF  IDEAS 

09-Roll  Up  Reports  ! 

102 

Communicate  and  Report 

f  ooeo  ‘ 

[fluency  OFIDiAS 

Vo-Call  to  Conference  | 

■Ea 

Communicate  and  Report 

I  0080 

IFLUENCY  OF  IDEAS 

26-Receive  Digital  Message  j 

■EH 

Communicate  and  Report 

j  0080 

IFLUENCY  OF  ID^S 

27-Input  data  Into  Computer  j 

105 

Communicate  and  Report 

1  bosb 

iF=LUENCY  OF  IDEAS 

28-Send  Digital  Information  \ 

106 

Communicate  and  Report 

j  ’bbbb”^ 

iFLEXiBILlfY  OF  CLOSURE 

61 -Receive  and  Record/Analog  ! 

107 

Communicate  and  Report 

1  0090 

[FLEXIBILITY  OF  CLOSURE 

i  02-Pass  Information  1 

0.00  I 
j  dJd""i_ 

"fo'oo'l 

']"'aoo\ 

'  {  MO 
I  0.00  i_ 
j  0.00  S 
j  0^00  [ 
b.ob  ! 

0.00  L 

'['■^26  I 

’]  "i26  [ 

"j . bTbb\ 

I'?!! 

j  0.00  ! 

“['o'bbT 

'l'''abo'”j_ 

'f'abbL 

j  abo  l_ 

IpML 
j  0  00  L 
T'abb\ 
j  "abF[ 
"l  bjbbT 

'1 . b7oT[ 

■fbTbb'L 

. j"'b.bb”r 

j  b.bb  |_ 

"T'o'bb'L 

. l’'b.oFi_ 

"1 . b.ob"l_ 

I  b.bb  L 

J . abb . i 

„„_■■■[ 

. j . abb . [ 

,1  b.bb'L 

"|"abb'""[ 

1 . b.bb' . L 

'"i . abb . |_ 

Tb;bb'"L 

’']”~abb'T 

I  b.bb  j 


1 . 0.00  1 

i  b.bb  I 
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B 

C 

D 

E 

F 

G 

H 

1 

1 

M: 

M 

I 

'SddS. 

•Mil 

108 

Communicate  and  Report 

0090 

FLEXIBILITY  OF  CLOSURE 

03-Listen-Receive  Information  1 

1.42 

1.20 

1.70 

Communicate  and  Report 

0090 

FLEXIBILITY  OF  CLOSURE 

04-Secondary  Monitor 

0.00 

1.20 

0.00 

110 

Communicate  and  Report 

0090 

FLEXIBILITY  OF  CLOSURE 

05-Log  Message 

0.00 

1.20 

0.00 

111 

Communicate  and  Report 

0090 

FLEXIBILITY  OF  CLOSURE 

06-Route  (Outside  the  Section) 

0.00 

1.20 

0.00 

MOM 

Communicate  and  Report 

0090 

FLEXIBILITY  OF  CLOSURi 

07-Send  Message 

0.00 

1.20 

0.00 

113 

Communicate  and  Report 

0090 

FLEXIBILITY  OF  CLOSURE!  08-Verbal  Order 

0.00 

1.20 

0.00 

114 

Communicate  and  Report 

0090 

FLEXIBILITY  OF  CLOSURE!  09-Roll  Up  Reports 

1.42 

1.20 

1.70 

115 

Communicate  and  Report 

0090 

FLExTBILifY  OF  CLOSURE!  ld-Cajl  to  Conference 

0.00 

1.20 

0.00 

116 

Communicate  and  Report 

0090 

FLEXIBILITY  OF  CLOSURE 

26-Receive  Digital  Message 

0.00 

1.20 

0.00 

117 

Communicate  and  Report 

0090 

FLEXIBILITY  OF  CLOSURE 

27-Input  data  Into  Computer 

0.00 

1.20 

0.00 

KITSl 

Communicate  and  Report 

0090  j 

FLEXIBILITY  OF  CLOSURE 

28-Send  Digital  Information 

0.00 

1.20 

0.00 

Communicate  and  Report 

0100 

SELECTIVE  ATTENTION 

01 -Receive  and  Record/Analog 

1.50 

1.20 

1.80 

Communicate  and  Report 

0100 

02-Pass  Information 

1  50 

1.20 

1.80 

1  121 

Communicate  and  Report 

0100 

03-Llsten-Receive  Information 

1.50 

1,20 

1.80 

Communicate  and  Report 

0100 

04-Secondary  Monitor 

150 

1.20 

1.80 

123 

Communicate  and  Report 

0100 

ISELEcfivE  Mention 

05-Log  Message 

1.50 

1.20 

1.80 

MFa 

Communicate  and  Report 

oi'od 

SELECTIVE  ATTENTION 

06-Route  (Outside  the  Section) 

d.dd 

1.20 

0.00 

125 

Communicate  and  Report 

6^66 

SELECTIVE  ATTENTION 

07-Send  Message 

1.50 

L 

1.20 

1.80 

126 

Communicate  and  Report 

0100 

SELECTIVE  ATTENTION 

08-Verbal  Order 

1.50 

1,20 

1.80 

Mvkm 

Cornmunicate  and  Report 

0100 

SELECfiVE  ATTENTION 

09-Roll  Up  Reports 

1.50 

u 

1.20 

1.80 

Communicate  and  Report 

0100 

SELECTIVE  ATTENTION 

10-Call  to  Conference 

0,00 

1.20 

0.00 

Communicate  and  Report 

olod 

SELECT  ivi  Aff  Eisif  ION 

26-Receive  Digital  Message 

1.50 

1.20 

1.80 

1  130 

Communicate  and  Report 

0100 

SELECTIVE  ATTENTION 

27-Input  data  Into  Computer 

1.50 

1.20 

1.80 

■Ml 

Communicate  and  Report 

0100 

SELECTIVE  ATTENTION 

28-Send  Digital  Information 

1,50 

1.20 

1.80 

132 

Communicate  and  Report 

0110 

SPATIAL  ORIENTATION 

01-Recelve  and  Record/Analog 

0.00 

1.20 

0.00 

133 

Communicate  and  Report 

'oiTo^l 

SPATIAL  ORIENTATION 

02-Pass  Information 

1  d[dd" 

1.20 

0.00 

134 

Comrnun^^ 

0110 

spatiaE  orientatTon 

03-Listen-Recelve  Information 

fabol 

1.20 

0.00 

135 

Communicate  and  Report 

om] 

SPATIAL  ORIENTATION 

04-Secondary  Monitor 

1  o.dd  ; 

1.20 

0.00 

Communicate  and  Report 

oTicn 

SPATIAL  ORIENTATION 

05-Log  Message 

I  0.00  i 

1.20 

0.00 

1  137 

Communicate  and  Report 

j  oiio 

SPATIAL  ORIENTATION 

06-Route  (Outside  the  Section) 

1 add] 

1.20 

0.00 

ri)iio"i 

SPATIAL  ORIENTATION 

07-Send  Message 

d.do  1 

1.20 

0.00 

139 

Comrnuni^ 

1  ofio 

SPATIAL  ORIENTATION 

08-Verbal  Order 

0.00  1 

1.20 

0,00 

140 

Commurii^^^^^^ 

i  0110  i 

SPATIAL  ORIENTATION 

09-Roil  Up  Reports 

"d.dd  i 

1.20 

0.00 

■EH 

Corrirnunicate  and  Report 

SPATIAL  ORIENTATION 

10-Call  to  Conference 

o.od  ! 

1.20 

0.00 

1  142 

Communicate  and  Report 

1  Olid' 

ORIEN  WION 

26-Receive  Digital  Message 

add  1 

1.20 

0.00 

Coipmuntete  and 

1 . diio'^" 

SPATIAL  ORiENTAfiON 

27-Input  data  Into  Computer 

d.do  1 

L 

1.20 

0.00 

■ES 

Communicate  and  Report 

1  'dTiF" 

SPATIAL  ORIENTATION 

28:Send  Digital 

“d[dd  1 

1,20 

0.00 

Communirate  a^ 

1 . di2r"i 

IvisUALIZATION 

01  -Receive 

did  ! 

1.20 

0.00 

Communicate  and  Report 

1  0120 

[viSUALii^flON“ 

02-Pass  Information 

d^l 

1.20 

0.00 

■Ea 

Communicate  and  Report 

[0120 . 

[visualization 

03-Llsten-Receive  Information 

6.66  1 

1.20 

0.00 

1  148 

Communicate  and  Report 

jdii^ . 

jvISUALI^flON .  . . 

d4-Secondary  Monitor 

0,00  1 

1.20 

0.00 

Communicate  and  Report 

[0120 

[visu/Iii^fioN 

05-Log  Message 

add  1 

1.20 

0.00 

Communicate  and  Report 

1  01^"~ 

IVISUALIZATION 

dd-Route  (Outside  the  Section)  j 

0.00 

j 

1.20 

0.00 

Communicate  and  Report 

i  0120 

[visualization 

d7-Send  Message 

i  d.od : 

1.20 

0.00 

Communicate  and  Report 

foi^' 

[visualization 

08-Verbal  Order 

1  o.do 

I” 

1.20 

0.00 

Communicate  and  Report 

[“0120 

IVISUALIZATION 

dd-Roii  Up  Reports 

1  0.00  i 

i 

1.20 

0,00 

Communicate  and  Report 

i  0120 

[visualization 

id-Call  to  Conference 

[  ojod  i 

U 

1.20 

0.00 

1  155 

Communicate  and  Report 

1  0120 

iVlSUAUi^fldN 

26-Receive  Digital  Message 

j  add  i 

I 

1.20 

0.00 

Communicate  and  Report 

1  oiTo 

IVISUALIZATION 

27-Input  data  Into  Computer 

|1add 

n 

1.20 

0,00 

Communicate  and  Report 

i . 0120 

iviSU^IZATION 

28-Send  Digital  Information 

i  0.00 

1.20 

0.00 

Communicate  and  Report 

j  "0130“' 

jlNDUCTIVE  REASONING 

di-Recelve  and  Record/Analog 

j  o.do 

I 

1.20 

0.00 

1  159 

Communicate  and  Report 

1  0130 

i INDUCTIVE  REASONING 

02-Pass  Information 

i  0.00 

1.20 

0.00 

|Kn!ii 

Communicate  and  Report 

!  0130 

jiNDUCfivi  p^gQ|sjji;;jQ 

dS-Listen-Receive  Information 

I aoo 

1.20 

0.00 
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B 

C 

D 

- E - 

F 

G 

H 

1 

1 

ft 

•  ■' 

0  te'l 

1 

Communicate  and  Report 

0130 

INDUCTIVE  REASONING 

04-Secondary  Monitor 

0.00 

1.20 

0.00 

\J62 

Communicate  and  Report 

0130 

INDUCTIVE  REASONING 

05-Log  Message 

0.00 

1.20 

0.00 

KE!cl 

Communicate  and  Report 

0130 

INDUCTIVE  REASONING 

06-Route  (Outside  the  Section) 

0.00 

1.20 

0.00 

■K1 

Communicate  and  Report 

0130 

INDUCTIVE  REASONING 

07-Send  Message 

0.00 

1.20 

0.00 

Communicate  and  Report 

0130 

INDUCTIVE  REASONING 

08-Verbal  Order 

6.00 

1,20 

0.00 

UKM 

Communicate  and  Report 

0130 

INDUCTIVE  REASONING 

09-Roll  Up  Reports 

0.00 

1.20 

0.00 

Communicate  and  Report 

0130 

INDUCTIVE  REASONING 

10-Call  to  Conference 

0.66 

1.20 

0.00 

[m 

Communicate  and  Report 

0130 

INDUCTIVE  REASONING 

26-Receive  Digital  Message 

0.00 

L. 

1.20 

0.00 

KESl 

Communicate  and  Report 

0130 

INDUCTIVE  REASONING 

27-Input  data  Into  Computer 

0.00 

1.20 

0.00 

LlTO 

Communicate  and  Report 

0130 

iNDUCflVE  REASONING 

28-Send  Digital  Information 

0.00 

1.20 

0.00 

KBl 

Communicate  and  Report 

0140 

CATEGORY  FLEXIBILITY 

01 -Receive  and  Record/Analog 

0.00 

1.20 

0.00 

172 

Communicate  and  Report 

0140 

CATEGORY  FLEXIBILITY 

02-Pass  Information 

0.00 

1.20 

0.00 

173 

Communicate  and  Report 

0140 

CATEGORY  FLEXIBILITY 

03-Listen-Receive  Information 

0.00 

1.20 

0.00 

174 

Communicate  and  Report 

0140 

CATEGORY  FLEXIBILITY 

04-Secondary  Monitor 

0.00 

1.20 

0.00 

175 

Communicate  and  Report 

0140 

FLixiBILITY 

05-Log  Message 

0.00 

1.20 

0,00 

176 

Communicate  and  Report 

0140 

CATEGORY  FLEXIBILITY 

06-Route  (Outside  the  Section) 

0.00 

1.20 

0.00 

MHM 

Communicate  and  Report 

0140 

CMEGORY  FLEXIBILITY 

07-Send  Message 

0.00 

1.20 

0.00 

178 

Communicate  and  Report 

oub 

08-Verbal  Order 

0.00 

1,20 

0.00 

Communicate  and  Report 

0140 

CATEGORY  FLEXIBILITY 

09-Roll  Up  Reports 

0.00 

1.20 

0.00 

Wi!il 

Communicate  and  Report 

0140 

CATEGORY  FLEXIBILITY 

10-Call  to  Conference 

0.66 

1.20 

0.00 

KEtl 

Communicate  and  Report 

0140 

CATEGORY  FLEXIBILITY 

26-Receive  Digital  Message 

0.00 

— 

1.20 

0.00 

1  182 

Communicate  and  Report 

0140 

category  flexibility 

27-Input  data  Into  Computer 

0.00 

1.20 

0.00 

Communicate  and  Report 

0140 

ICME^F^  FL^IBiLITY 

28-Send  Digital  Information 

0,00 

1.20 

0.00 

Communicate  and  Report 

0150 

DEDUCTIVE  REASONING 

01 -Receive  and  Record/Analog 

0,00 

1.20 

0.00 

Communicate  and  Report 

0150 

DEDUCTIVE  REASONING 

02-Pass  Information 

0.00 

1.20 

0.00 

1  186 

Communicate  and  Report 

j . 01 50"" 

03-Listen-Receive  Information 

T661 

1.20 

0.00 

C^omiTiup^ 

. 01^~ 

DiDUCfiVER^^^^ 

04-Secondary  Monitor 

ISW 

1.20 

0.00 

T!S1 

Commu  and  Report 

0150 

deductive  REASONS 

05-Log  Message 

0.00  ^ 

1.20 

0.00 

189 

Communicate  and  Report 

"oTso  ’ 

DEDUCTIVE  REASONING 

06-Rqute  (Outside  the  Section) 

[6.661 

1.20 

0,00 

190 

j  "bl50~j 

lOTrGend  Mi^sage 

1 oW 

1 

1.20 

0.00 

191 

Communicate^^a 

r’'bi5o"] 

DEDUCfiVE  REASONING  1 

08-Verbai  Order 

r6'66 

\ _ 

1.20 

0.00 

mm 

Commuri^^^^^^ 

1 . bi5b'”| 

DEDUCTIVE  REASONING  1 

I  Oy^li^Up  Re);»rte^^  j 

i  0.66 

i 

1.20 

0.00 

W3tl 

Commu^^^  , 

1  bi'sb"  1 

i  DEDUCTIVE  l^SON 

ip-Ca!l  to  C^^  ; 

[6166“ 

1.20 

0.00 

Kligl 

Communicate 

”  oTso’] 

iDEbucf^VER^S^^^ 

1 26-Receive  Digital  Message 

[6166^ 

r 

1.20 

0.00 

KEa 

Communicate  and  Report  i 

riiT&b“| 

|27-lnput  data  into 

6.00 

1.20 

0.00 

Comrnunicate^ 

28-Send 

[  o^oo 

1.20 

0.00 

1  197 

Communicate  an^^^ 

rbieb'l 

lINFORMATiON  ORDERING 

01 -Receive  and  Record/Analog 

1  0.00 

s 

1.20 

0.00 

BiTil 

INFORMATION  ORDERING  02-Pass  Information 

6.66 

i 

1.20 

0.00 

BUM 

Communicate  and  Report  | 

r'blio": 

INFORMATION  ORDERING 

03-Listen-Receive  Information 

6.66 

P 

! _ 

1.20 

0.00 

1  200 

Communicate  and  Report 

f~bl60”^ 

INFORMATION  ORDERING  04-Secondary  Monitor 

0.66' 

1 

1.20 

0.00 

ESI 

Communicate  and  Report  j 

j  "bieb'" 

INFORMATION  ORDERINC^  05-Log  Message 

0.00 

L 

1.20 

0.00 

1  202 

Communicate  and  Report 

j  “bi6b“^ 

INFORMATION  ORDERING 

06-Route  (Outside  the  Section) 

1  o“.o6 

L 

1.20 

0.00 

Kikl 

Communicate  and  Report 

'"bibb”’ 

INFORMATION  ORDERIN(^07-Send  Message 

bob 

1.20 

0.00 

WilUM 

Communicate  and  Report 

■'bibb” 

INFORMATION  ORDERING!  08-Verbal  Order 

j  bob 

i 

1.20 

0.00 

K>a 

Communicate  and  Report 

6i6'6 

INFORMATION  ORDERING  09-Roll  Up  Reports 

1  3.08 

1 

1.20 

3.70 

1  206 

Communicate  and  Report 

0160 

INFORMATION  ORDERING  10-Call  to  Conference 

rb.bb” 

i 

1.20 

0.00 

Kia 

Communicate  and  Report 

0160 

INFORMATION  ORDERING  26-Receive  Digital  Message 

I'bbb 

1.20 

0.00 

208 

Communicate  and  Report 

1  0160 

INFORMATION  ORDERING  27-Input  data  Into  Computer 

fb’bb 

1.20 

0.00 

209 

Communicate  and  Report 

I  0160 

^INFORMATION  ORDERING 28-Send  Digital  Information 

1  0.00 

1.20 

0.00 

EPl 

Communicate  and  Report 

1  0170 

jMATHEMATICAL  REASONI 

01 -Receive  and  Record/Analog 

i . abb" 

L 

1.20 

0.00 

Communicate  and  Report 

i  0170 

SMATHEMATICAL  REASONI 

02-Pass  Information 

1  0.00 

1 _ 

1.20 

0.00 

212 

Communicate  and  Report 

f  0170” 

1  MATHEMATICAL  REASONI 

03-Listen-Receive  Information 

[bob 

1.20 

0.00 

213 

Communicate  and  Report 

i  0170 

iMATHEMATICAL  REASONI 

04-Secondary  Monitor 

1  0.00 

1.20 

0.00 
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B  I  C  I  D  I  E  1 

F 

I 

H 

1 

1 

9 

K 

214 

Communicate  and  Report 

0170 

MATHEMATICAL  REASON!  05-Log  Message 

0.00 

1.20 

0.00 

215 

Communicate  and  Report 

0170 

MATHEMATICAL  REASON!  06-Route  (Outside  the  Section) 

0.00 

1.20 

0,00 

Communicate  and  Report 

0170 

MATHEMATICAL  REASON!  07-Send  Message 

0.00 

1.20 

0.00 

Communicate  and  Report 

0170 

MATHEMATICAL  REASON II  08-Verba!  Order 

0.00 

1.20 

0.00 

Communicate  and  Report 

0170 

MATHEMATICAL  REASONI 

09-Roll  Up  Reports 

2.00 

1.20 

2.40 

Communicate  and  Report 

0170 

MATHEMATICAL  REASONI 

10-Call  to  Conference 

0.00 

1.20 

0.00 

j^SI 

Communicate  and  Report 

0170 

MATHEMATICAL  REASONI 

26-Receive  Digital  Message 

0.00 

1.20 

0.00 

221 

Communicate  and  Report 

0170 

MATHEMATICAL  REASON ll  27-Input  data  Into  Computer 

0.00 

1.20 

0.00 

222 

Communicate  and  Report 

0170 

MATHEMATICAL  REASONI 

28-Send  Digital  Information 

0.00 

1.20 

0.00 

223 

Communicate  and  Report 

0180 

NUMBER  FACILITY 

01 -Receive  and  Record/Analog 

0.00 

1.20 

0.00 

Communicate  and  Report 

0180 

number  FACILiTY 

02-Pass  Information 

0.00 

1.20 

0.00 

Communicate  and  Report 

0180 

NUMBER  FACILITY 

03-Llsten-Receive  information 

6.00 

1.20 

0.00 

Communicate  and  Report 

0180 

NUMBER  FACILITY 

04-Secondary  Monitor 

0.00 

1.20 

0.00 

Communicate  and  Report 

0180 

NUMBER  FACILITY 

05-Log  Message 

6.00 

1.20 

0.00 

Communicate  and  Report 

0180 

NUMBER  FACILITY 

06-Route  (Outside  the  Section) 

0.00 

1.20 

0.00 

Communicate  and  Report 

0180 

NUMBER  FACILITY 

07-Send  Message 

0.00 

L 

1.20 

0.00 

230 

Communicate  and  Report 

0180 

NUMBER  FACILITY 

08-Verbal  Order 

0.00 

1.20 

0.00 

231 

Communicate  and  Report 

0180 

NUMBER  FACILITY 

09-Ro!l  Up  Reports 

2.50 

1.20 

3.00 

Communicate  and  Report 

0180 

NUMBER  FACILITY 

10-Call  to  Conference 

0.00 

1.20 

0.00 

Communicate  and  Report 

0180 

NUMBER  FACILITY 

26-Receive  Digital  Message 

0.00 

1.20 

0.00 

Communicate  and  Report 

0180 

NUMBER  FACILITY 

27-Input  data  Into  Computer 

0.00 

1.20 

0.00 

Communicate  and  Report 

0180 

NUMBER  FACILITY 

28-Send  Digital  Information 

0.00 

1.20 

0,00 

Communicate  and  Report 

0190 

TIME  SHARING 

01 -Receive  and  Record/Analog 

2.00 

1.20 

2,40 

237 

Communicate  and  Report 

0190 

TIME  SHARING 

02-Pass  Information 

0.00 

1.20 

0.00 

238 

Communicate  and  Report 

0190 

TIME  SHARING 

03-Listen-Receive  Information 

0.00 

1.20 

0.00 

Communicate  and  Report 

0190 

TIME  SHARING 

04-Secondary  Monitor 

2.00 

1.20 

2.40 

1  240 

Communicate  and  Report 

0190 

tTme  sharing 

05-Log  Message 

0.00 

1.20 

0.00 

moM 

Communicate  and  Report 

0190 

TIME  SHARING 

06-Route  (Outside  the  Section) 

0.00 

1.20 

0.00 

K3fM 

Communicate  and  Report 

0190 

TIME  SHARING 

07-Send  Message 

0.00 

1.20  1 

0.00 

Communicate  and  Report 

1  0190  1 

fiME  SHARING 

08-Verbal  Order 

0.00  1 

L 

1.20 

0.00 

244 

Communicate  and  Report 

[  "  0190  '1 

TIME  SHARING 

09-Roll  Up  Reports 

6.66  \ 

1.20 

0.00 

245 

^mmi^ii^te  and 

1 . ~6m  1 

TIME  SHARING 

10-Call  to  Conference 

0.00  i 

1.20  : 

0.00 

E3 

Comrnunicate  and  Report 

r^^oi96 . i 

TIME  SHARING 

26-Receive  Digital  Message 

. ioo . i 

1.20 

2.40 

Coipmuriiratea 

1  Q^QQ 

TIME  SHArInG 

27-Input  data  Into  Computer 

. 2M . 1 

1.20 

2.40 

1  248 

Communicate  and  Report  i 

[0190^^ 

TIME  SHARING 

28-Send  Digital  Information 

''iool 

L 

1.20  ! 

2.40 

Communicate  and  Report  ! 

[  0260 ’ 

SPEED  OF  CLOSURE 

01 -Receive  and  Record/Analog 

. 6"oo  1 

L 

1.20 

0.00 

1  250 

Communicate  and  Report  | 

r~62oo'^ 

|SPEED  OF  CLOSURE 

02-Pass  Information 

j  0.00  ! 

i _ 

1.20 

0.00 

KiW 

Communicate  and  Report 

r  0200 . 

SPEED  OF  CLOSURE 

03-Llsten-Receive  Information 

1 . aoo”'' 

1.20 

0.00 

252 

Communicate  and  Report  | 

I  0200 

04-Secondary  Monitor 

1 . o'oo": 

1.20 

0.00 

Communicate  and  Report  1 

j  0206 

speed  of  closure 

05-Log  Message  I 

l  “o;oo  ‘ 

j _ 

1.20 

0.00 

1  254 

Comrriunicate  an^  Repo^  J 

--Q26F' 

SPEED  OF  CLOSURE 

06-Route  (Outside  the  Section)  1 

1  0.00 

1.20 

0.00 

mam 

0200 

SPEED  OF  CLOSURE 

07-Send  Message 

i  6.oo~ 

1.20 

0.00 

Communicate  and  Report  1 

iD200"  ! 

SPEED  OF  closure 

08-Verbal  Order 

r  aoo 

L_ 

1.20 

0.00 

1  257 

Communicate  and  Report 

0200 

SPEED  OF  CLO^RE 

09-Roll  Up  Reporte 

\6j66 

1.20 

0.00 

Communicate  and  Report 

0200  1 

SPEED  OF  CLOSURE 

10-Call  to  Conference 

1  6.00 

1.20 

0.00 

Communicate  and  Report 

0200 

SPEED  OF  CLOSURE 

26-Receive  Digital  Message 

foToo  ‘ 

1.20 

0.00 

Comrnunic^^^ 

0200 

SPEED  OF  CLOSURE 

27-Input  data  Into  Computer 

i  0.06 

1.20 

0.00 

1  261 

Communicate  and  Report 

0200 

speed  of  closure 

28’Send  Digital  Information 

1  6.66 

1.20 

0.00 

Communicate  and  Report 

'02t0 

^PERCEPTUAL  SPEED  AND 

01 -Receive  and  Record/Analog 

j  0.00 

1.20 

0.00 

■eiaca 

Communicate  and  Report 

. 0210 

PERCEPTUAL  SPEED  AND!  02-Pass  Information 

i oM 

t 

1.20 

0.00 

Communicate  and  Report 

0216 

PERCEPTUAL  SPEED  AND03-Listen-Receive  Information 

\  aoo 

1 

1.20 

0.00 

EH 

i  0210 

PERCEPTUAL  SPEED  AND  04-Secondary  Monitor 

i  0.00 

i 

1.20 

0.00 

I  266 

Communicate  and  Report 

1  0210 

[PERCEPTUAL  SPEED  ANO  05-Log  Message 

i  0.00 

1.20 

0.00 
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B 

C 

D 

E  - 1 

:t::i 

H 

1 

1 

S 

B 

Communicate  and  Report 

0210 

PERCEPTUAL  SPEED  AN[^  06-Route  (Outside  the  Section) 

0.00 

1.20 

0.00 

I  268  I 

Communicate  and  Report 

0210 

perceptual  speed  ANd07-Send  Message 

0.6b 

120 

0.00 

Communicate  and  Report 

0210 

PERCEPTUAL  SPEED  ANojoS-Verbal  Order 

0.00 

1.20 

0.00 

Communicate  and  Report 

0210 

F>ERCEPTUAL  speed  ANC^OO-RoII  Up  Reports 

2.38 

1.20 

2.86 

271 

Communicate  and  Report 

0210 

PERCEPTUAL  SPEED  ANOj  10-Call  to  Coriference 

0.00 

1.20 

0.00 

272 

Communicate  and  Report 

0210 

PERCEPTUAL  SPEED  ANI^26-Receive  Digital  Message 

0.00 

120 

0.00 

273 

Communicate  and  Report 

0210 

PERCEPTUAL  SPEED  ANd 

27-Input  data  Into  Computer 

0.00 

120 

0.00 

274 

Communicate  and  Report 

0210 

p— — ~ 

28-Send  Digital  Information 

0.00 

120 

0.00 

Communicate  and  Report 

0220 

REACTION  TIME 

01 -Receive  and  Record/Analog 

0.00 

120 

0.00 

276 

Communicate  and  Report 

0220 

REACTION  TIME 

02-Pass  Information 

0.00 

1.20 

0.00 

277 

Communicate  and  Report 

0220 

REACTION  TIME 

03-Listen-Recelve  Information 

0.00 

1.20 

0.00 

Communicate  and  Report 

0220 

REACTION  TIME 

04-Secondary  Monitor 

0.00 

1.20 

0.00 

^1 

Communicate  and  Report 

0220 

REACTION  TIME 

05-Log  Message 

0.00 

1.20 

0.00 

280 

Communicate  and  Report 

0220 

REACTION  TIME 

06-Route  (Outside  the  Section) 

6.06 

1.20 

0.00 

281 

Communicate  and  Report 

0220 

REACTION  TIME 

07-Send  Message 

0.00 

1.20 

0.00 

wm 

Communicate  and  Report 

0220 

REACTION  TIME 

08-Verbal  Order 

0.00 

1.20 

0.00 

Communicate  and  Report 

0220 

reaction  time 

09-RoII  Up  Reports 

0.00 

120 

0.00 

wm 

Communicate  and  Report 

0220 

REACTION  fiME 

10-Call  to  Conference 

0.00 

1.20 

0.00 

Communicate  and  Report 

0220 

REACTION  TIME 

26-Receive  Digital  Message 

0.00 

120 

0.00 

wm 

Communicate  and  Report 

0220 

REACTION  TIME 

27-Input  data  Into  Computer 

0.00 

120 

0.00 

mm 

Communicate  and  Report 

0220 

REACTION  TIME 

28-Send  Digital  Information 

0.00 

120 

0.00 

288 

Communicate  and  Report 

0230 

CHOICE  REACTION  TIME 

01 -Receive  and  Record/Analog 

1  0.00 

1.20 

0.00 

289 

Communicate  and  Report 

0230 

CHOICE  REACTION  TIME 

02-Pass  Information 

6.66 

120 

0.00 

290 

Communicate  and  Report 

0230 

CHOICE  REACTION  TIME 

03-Llsten-Receive  Information 

6.66 

1.20 

0.00 

Communicate  and  Report 

0230 

CHOICE  REACTION  TIME 

04-Secondary  Monitor 

6.66 

120 

0.00 

Communicate  and  Report  ; 

0230 

CHOICE  REACTION  TIME 

05-Log  Message 

0.00 

120 

0.00 

Kilcl 

Communicate  and  Repo^^^ 

ro2^ 

iCHOICE  REACTION  TIME 

06-Route  (Outside  the  Section) 

0.00 

120 

0.00 

1  294 

\  0230 

CHOICE  REACTION  f  IME 

07-Send  Message 

6.66 

120 

0.00 

0230 

CHOICE  REACTION  TIME 

08-Verbal  Order 

0.00 

1.20 

0.00 

296 

Communicate  and  Report 

02^ 

CHOICE  REACTION  TIME 

09-Roll  Up  Reports 

aoo ! 

1.20 

0.00 

297 

1  0^ 

CHOICE  REACTION  TIME 

10-Call  to  Conference 

6.06 

’ 

1.20 

0.00 

298 

Communicate  and  Report 

. 02^ . i 

CHOICE  REACTION  TIME 

26-Receive  Digital  Message 

-666 

1.20 

0.00 

Cornmunlcate  a^^ 

r^^oi^o . ! 

CHOICE  REACTION  TIME 

27-Input  data  Into  Computer 

QW 

120 

0.00 

300 

Cornmunlca^^^^^^ 

. 0230  \ 

CHOICE  REACTION  TIME 

28-Send  Digital  Information 

6166' 

1.20 

0.00 

Communicate  and  Report 

0240 

NEAR  VISION 

01 -Receive  and  Record/Analog 

(1561 

E _ 

1.20 

1.51 

302 

Communicate  and  Report 

0240 

jNEAR  VISION 

02-Pass  Information 

126 . 

1.20 

1.51 

303 

Communicate  and  Report 

6240^ 

[NEAR  VlisiON'^ . 

03-Llsten-Receive  Information 

. i56''I 

1.20 

151 

304 

Communicate  and  Report  1 

1 . 0240 

iN^RVlSION 

04-Secondary  Monitor 

1.26 

< 

L 

120 

151 

305 

Communicate  and  Report  i 

ii^RvisioN 

05-Log  Message 

. "i"56 . 1 

120 

151 

306 

Communicate  and  Report  | 

1  0^0 

INEAR  VISION 

0.00 

1.20 

0.00 

307 

Cqrnrnu nlc^te  a^  . i 

1 . 0240  i 

INEAR  VISION 

67-Send  Message 

"'"lie"'! 

120 

151 

308 

!  0240 

[near  VISION 

08-Verbal  Order 

126 

120 

151 

309 

Communicate  and  Report  j 

r  0^0  ^ 

jNEAR  VISION 

09-Roll  Up  Reports 

. 156 . 1 

1.20 

1.51 

EIEI 

[NEAR  VISION 

6.00 

1.20 

0.00 

311 

Communicate  and  Report  1 

\  0240 

jNEAR  vislON . 

26-Receive  Digital  Message 

126“! 

i“ 

1.20 

151 

312 

Communicate  and  Report  ! 

0240 

INEAR  VISION 

27-Input  data  Into  Computer 

1.26 

1.20 

151 

EI9 

Corpmunicate  and  Report  i 

0240 

[near  VISION 

28-Send  Digital  Information 

1 i56 

1.20 

151 

314 

Communicate  and  Report 

0250 

|FAR  VISION 

01-Receive  and  Record/Analog 

0.00 

1.20 

0.00 

315 

Comrpunicate  and  Re^^  J 

!  0250 

1  FAR  VISION 

02-Pass  Information 

0.00 

1.20 

0.00 

mm 

Communicate  and  Report  i 

r0250'" 

[FAR  VISION 

03-Listen-Receive  Information 

0.00 

l_ 

120 

0.00 

317 

Communicate  and  Report  1 

i  0250 

[far  VISION 

b4-Secondary  Monitor 

1  0.00 

1.20 

0.00 

318 

Communicate  and  Report 

1  0250 

I  FAR  VISION 

j  05-Log  Message 

1  656 

p 

1.20 

0.00 

319 

Communicate  and  Report 

roiso'^ 

IFAR  VISION 

j  06-Route  (Outside  the  Section) 

1.50  I 

120 

1.80 
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B 

C 

D 

_ i _ IF 

G 

H 

1 

1 

mm 

Mil 

Communicate  and  Report 

0250 

FAR  VISION 

07-Send  Message 

0.00 

1.20 

0.00 

321 

Communicate  and  Report 

0250 

FAR  VISION 

08-Verbal  Order 

0.00 

1.20 

0.00 

322 

Communicate  and  Report  | 

0250 

FAR  VISION 

09-Roll  Up  Reports 

0.00 

1.20 

0.00 

Communicate  and  Report 

0250 

FAR  VISION 

10-Call  to  Conference 

0.00 

1.20 

0.00 

324 

Communicate  and  Report 

0250 

FAR  VISION 

26-Receive  Digital  Message 

0.00 

1.20 

0.00 

325 

Communicate  and  Report 

0250 

FAR  VISION 

27-Input  data  Into  Computer 

0.00 

1.20 

0.00 

Communicate  and  Report 

0250 

FAR  VISION 

28-Send  Digital  Information 

0.00 

1.20 

0.00 

Communicate  and  Report 

0260 

NIGHT  VISION 

01 -Receive  and  Record/Analog 

0  00 

1.20 

0.00 

328 

Communicate  and  Report 

0260 

NIGHT  VISION 

02-Pass  Information 

0.00 

1.20 

0.00 

Communicate  and  Report 

0260 

NIGHT  VISION 

03-Listen-Receive  Information 

0.00 

1.20 

0.00 

Communicate  and  Report 

0260 

NIGHT  VISION 

04-Secondary  Monitor 

0.00 

1.20 

0.00 

331 

Communicate  and  Report 

0260 

NIGHT  VISION 

05-Log  Message 

0.00 

1.20 

0.00 

332 

Communicate  and  Report 

0260 

NIGHT  VISION 

06-Route  (Outside  the  Section) 

..  0,00  _ 

1.20 

0.00 

333 

Communicate  and  Report 

Q2gQ 

NIGHT  VISION 

07-Send  Message 

0.00 

1.20 

0.00 

334 

Communicate  and  Report 

0260 

NIGHT  VISION 

08-Verbal  Order 

0.00 

1.20 

0.00 

335 

Communicate  and  Report 

0260 

NIGHT  VISION 

09-Roll  Up  Reports 

0.00 

1.20 

0.00 

Communicate  and  Report 

0260 

NIGHT  VISION 

10-Call  to  Conference 

0.00 

1.20 

0.00 

1  337 

Communicate  and  Report 

0260 

NIGHT  visiON 

26-Receive  Digital  Message 

6.00 

1.20 

0.00 

IMM-M 

Communicate  and  Report 

0260 

NIGHT  VISION 

27-Input  data  Into  Computer 

0.00 

L 

1.20 

0.00 

339 

Communicate  and  Report 

0260 

NIGHT  VISION 

28-Send  Digital  Information 

0.00 

1.20 

0.00 

340 

Communicate  and  Report 

0275 

VISUAL  COLOR  DISCRIMIN  01 -Receive  and  Record/Analog 

0.00 

1.20 

0.00 

Communicate  and  Report 

0270 

VISUAL  COLOR  DISCRiMINi 02-Pass  ln?ormation 

0.00 

1.20 

0.00 

1  342 

Communicate  and  Report 

0270 

VISUAL  COLOR  D^SCRIMI^03-Llsten-Receive  Information 

0.00 

1.20 

0.00 

Communicate  and  Report 

0270 

VISUAL  COLOR  DISCRIMlN04-Secondary  Monitor 

0.00 

1.20 

0.00 

344 

Communicate  and  Report 

0270 

vi'sUAL  COLOR  DISCRIM iR 05-Log  Message 

6.66 

1.20 

0.00 

345 

Communicate  and  Report  j 

0270  J 

VISUAL  COLOR  DISCRIMIhlO^Route  (Outside  uie  Section) 

0.00 

1.20 

0.00 

Communicate  and  Report  | 

1  0270 

VISUAL  COLOR  DISCRIMlH07-Send  Message 

0.00 

1.20 

0.00 

347 

Communi^^  Report  i 

0270 

visUAL  COLOR  bisCRIMINloS-Verbal  Order 

0.00 

1.20 

0.00 

348 

Communicate  and  Report  ; 

0270 

ViSUAL  COLOR  DiSCRIMIh 

09-Roll  Up  Reports 

0.00 

1.20 

0.00 

349 

^mmunicj^e  and  ^  i 

0270 

ViSUAL  COLOR  DISCRIMIh 

10-Call  to  Conference 

6^6 

\ 

1,20 

0.00 

350 

Communicate  and  Report 

i  0270 

VISUAL  COLOR  DISCRIMI^ 

26-Receive  Digital  Message 

6.66 

h 

1.20 

0.00 

351 

Communicate  and  Report  i 

i  027^ 

ViSUAL  COLOR  DISCRIMI^ 

27-Input  data  Into  Computer 

!  1.50 

L 

1.20 

1.80 

jcgg 

Communicate  and  Report  i 

0270  jVISUAL  COLOR  DISCRIMI^ 

28-Send  Digital  Information 

. ll50  ' 

s 

120 

1.80 

■CW 

Communicate 

0280  jpiRIPHiFWkL  vTsiON 

01 -Receive  and  Record/Analog 

6"66 

! 

1.20 

0.00 

Cornmuntote  and^  J 

0280  fplRlPHEF5\LVrS 

02-Pass  Information 

1  6!66 

5 

1.20 

0.00 

Communicate 

0280  IPERIPHERAL  VISION 

03-Listen-Receive  Information 

1666 

1.20 

0.00 

Communicate  and  Report 

0280  jPERiPHEt^LVIsiON 

04-Secondary  Monitor 

1 6W 

1.20 

0.00 

Communicate  and  Report 

0280  [PERIPHERAL  VISION 

05-Log  Message 

6.00 

1.20 

0,00 

gld;ll 

Communicate  and  Report 

0280  IPERIPHERAL  VISION 

06-Route  (Outside  the  Section) 

0.00 

i 

1.20 

0,00 

■cw 

Communicate  and  Report 

0280  (PERIPHERAL  VISION 

07-Send  Message 

0.00 

1.20 

0.00 

1  360 

j^rnmunic^^ 

0280  IPERIPHERAL  VISION 

08-Verbal  Order 

!  0.00 

1 

1.20 

0.00 

gai 

0280  [PERIPHERAL  VISION 

09-Roli  Up  Reports 

0.00 

F 

!_ 

1.20 

0.00 

0280  IPERIPHERAL  VISION 

10-Call  to  Conference 

0.00 

1.20 

0.00 

g?i!cl 

Communicate  and  Report 

0280  1  PERIPHERAL  VISION 

26-Receive  Digital  Message 

!  0.00 

1.20 

0.00 

g^ 

Communicate  and  Report 

0280  jpERIPHERAL  VISION 

27-Input  data  Into  Computer 

0.00 

1.20 

0.00 

gKilW 

Communicate  and  Report 

0280  {PERIPHERAL  VISION 

28-Send  Digital  Information 

666 

1.20 

0.00 

gia;^ 

Communicate  and  Report 

0290  iDEPTH  PERCEPTION 

01 -Receive  and  Record/Analog 

0,00 

r 

1.20 

0.00 

gfsa 

Communicate  and  Report 

0290  {DEPTH  PERCEPTION 

02-Pass  Information 

om 

1.20 

0.00 

368 

Communicate  and 

0290  1 DEPTH  PERCEPTION 

03-Listen-Receive  Information 

666 

1.20 

0.00 

369 

0290  {DEPTH  PERCEPTION 

04-Secondary  Monitor 

j  0.00 

1,20 

0.00 

Communicate  and 

0290  IDEPTH  PERCEPfiON 

05-Log  Message 

1  0.00 

1 

1.20 

0.00 

371 

Communicate  and  Report 

0290  [DEPTH  PERCEPTION 

06-Route  (Outside  the  Section) 

1  1.00 

r 

1.20 

1.20 

372 

Communicate  and  Report 

0290  IDEPTH  PERCEPTION 

07-Send  Message 

1  0.00 

1.20 

0.00 
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B 

C 

D 

E 

F 

G 

H 

1 

1 

tjpetE# 

aWiI 

373 

Communicate  and  Report 

0290 

DEPTH  PERCEPTION 

08-Verbal  Order 

0.00 

120 

0.00 

374 

Communicate  and  Report 

0290 

DEPTH  PERCEPTION 

09-Roll  Up  Reports 

6.66 

120 

0.00 

375 

Communicate  and  Report 

0290 

DEPTH  PERCEPTION 

10-Call  to  Conference 

6.06 

120 

0.00 

376 

Communicate  and  Report 

0290 

DEPTH  PERCEPTION 

26-Receive  Digital  Message 

6.00 

1.20 

0.00 

IcCT 

Communicate  and  Report 

0290 

DEPTH  PERCEPTION 

27-Input  data  Into  Computer 

0.00 

120 

0.00 

Communicate  and  Report 

0290 

DEPTH  PERCEPTION 

28-Send  Digital  Information 

0.00 

120 

0.00 

Communicate  and  Report 

0300 

GLARE  SENSITIVITY 

01 -Receive  and  Record/Analog  | 

0.00 

1.20 

0.00 

Communicate  and  Report 

0300 

GLARE  SENSITIVITY 

02-Pass  Information 

0.00 

1.20 

0.00 

■clM 

Communicate  and  Report 

0300 

GLARE  SENSITIVITY 

03-Listen-Receive  Information 

0.00 

120 

0.00 

Communicate  and  Report 

0300 

GLARE  SENSITIVITY 

04-Secondary  Monitor 

0.00 

120 

0.00 

igf$l 

Communicate  and  Report 

0300 

GLARE 

05-Log  Message 

0.00 

120 

0.00 

384 

Communicate  and  Report 

0300 

GLARE  SENSITIVITY 

06-Route  (Outside  the  Section) 

0.00 

1.20 

0.00 

385 

Communicate  and  Report 

0300 

GLARE  SENSITIVITY 

67-Sencl  Message 

0.00 

1.20 

0.00 

386 

Communicate  and  Report 

0300 

GLARE  SENSITIVITY 

08-Verbal  Order 

0.00 

1.20 

0.00 

387 

Communicate  and  Report 

0300 

GLARE  SENSITIVITY 

OO-FRoll  Up  Reports 

6.66 

120 

0.00 

388 

Communicate  and  Report 

0300 

GLARE  SENSITIVITY 

10-Call  to  Conference 

0.00 

120 

0.00 

Communicate  and  Report 

0300 

GLARE  SENSITIVITY 

26-Receive  Digital  Message 

0.00 

120 

0.00 

I  390 

Communicate  and  Report 

0300 

GLARE  SENSITIVITY 

27-Input  data  Into  Computer 

0,00 

120 

0.00 

ESI 

Communicate  and  Report 

0300 

GLARE  SENSITIVITY 

28-Send  Digital  Information 

0.00 

1.20 

0.00 

392 

Communicate  and  Report 

0310 

GENERAL  HEARING 

01 -Receive  and  Record/Analog 

6.66 

1.20 

0.00 

Communicate  and  Report 

0310 

GENERAL  HEARING 

02-Pass  Information 

6.66 

1.20 

0.00 

Communicate  and  Report 

0310 

GENERAL  HEARING 

03-Listen-Receive  Information 

0.00 

120 

0.00 

Communicate  and  Report 

6310 

(^EF^L  HIRING 

04-Secondary  Monitor 

0.00 

1.20 

0.00 

396 

Communicate  and  Report 

0310 

GENERAL  HEARING 

05-Log  Message 

0.00 

120 

0.00 

397 

Communicate  and  Report 

0310 

iGENERAL  HEARING 

06-Route  (Outside  the  Section) 

0.00 

120 

0-00 

398 

Communicate  and  Report 

“0310  ! 

GENEI^^L  HEARING 

07-Send  Message 

^oW 

120 

0.00 

399 

Communicate  and  Report 

0310  1 

GENEF^LH^^ 

_  J 

0.00  \ 

120 

0.00 

400 

Communicate  and  Report 

0310” ! 

GENeSSiL  HEARING 

69-Roll  Up  Report 

0.00  i 

1.20 

0.00 

EiM 

Communicate  and  Report 

ozW] 

GENEF^L  HEARING 

10-Call  to  Conference 

0.06 

1.20 

0.00 

1402! 

Communicate  and  Report 

0310  j 

IGENEf6\L  hlEAFRING 

aoo 

120 

0.00 

Eiicl 

Communicate  and  Report 

0316'  i 

GENERAL  HIRING 

27-input  data  Into  (Computer 

0.00  ; 

! 

1.20 

0.00 

1  404 

Communicate  and  Report 

0310  i 

general  hearing . 

28^Send  DigiteMr^rm^ 

r  6.66  : 

1.20 

0.00 

Comrnunic^te  i 

”'“¥326'' . i 

... 

01  -Receive  and  FRecprd/Arialog 

\1W\ 

! 

1.20 

2.36 

Communicate  and  Report 

. 03^'''"] 

62;Pass  Infor^^  _ _ J 

6,66 

1.20 

0.00 

mitrm 

Commurtic^^^ 

1 . 0320"  1 

OO-Usten-R 

1.97  1 

r 

1.20 

2.36 

408 

Comrnuriicate  and  Report 

0320 

AUDiTOi^AffEN 

64-Secondary  Monitor . 

197 

1.20 

2.36 

409 

Comrpum 

“6326  i 

AUDITORY  ATTENTION 

. 

65-Log^Mes^^  . 

Too 

120 

0.00 

giM 

Communicate  and  Report 

0320  1 

Oe-Route  (Outside  the  Sectiorij 

066' 

1.20 

0.00 

wsn 

Comm^unicate  an 

0^ 

lAUDlfORY  AftiNfiON 

07-Sencl  Message 

oH”' 

! 

1.20 

0.00 

412 

^mmunic^  and  ^port  j 

1 . 6320 

[AUDITORY  ATTENTION 

OS-Verbal  Order 

oT) 

L 

1.20 

0.00 

413 

Communicate  and  Report 

1 . 0320 . ^ 

AUDITORY  ATTENTION 

69-Roii  Up  Reports 

0.00 

i 

120 

0.00 

EEl 

Comrnunicate  and  Report 

1  0320 

/i^DiTORY^ 

16-Cail  to  Conference 

. 197 . 1 

L 

120 

2.36 

1415 

Communicate  and  Report 

auditofry  attention 

26-Receive  Digital  Message 

. 197 

120 

2.36 

milM 

Communicate  and  Report 

[0326”' 

AUDITORY  ATTENTION 

27-input  data  Into  Computer 

6T) 

if 

120 

0.00 

417 

Communicate  and  Report 

i  AUDlfOF^  ATTENTION 

28-Send  Digital  Information 

a66 

1.20 

0.00 

418 

Communicate  and  Report 

0330 

ISOUND  LOCALIZATION 

61 -Receive  and  Record/Analog 

6.66 

S 

1.20 

0.00 

mlVM 

Communicate  and  Report 

0330 

SOUND  LOCALIZATION  i  02-Pass  Information 

6.66" 

n 

1.20 

0.00 

420 

Communicate  and  Report 

0330 

iSOUND  LOCALIZATION  OO-Listen-Receive  Information 

[666 

1.20 

0.00 

421 

Communicate  and  Report 

. 0330 

ISOUND  LOCALIZATION  1 04-Secondary  Monitor 

0.00 

120 

0.00 

Communicate  and  Report 

I  0330 

SOUND  LOCALIZATION  i05-Log  Message 

1  0.00 

\ 

1.20 

0.00 

423 

Communicate  and  Report 

j  0330 

; SOUND  LOCALIZATION  j  06-Route  (Outside  the  Section) 

666 

L 

120 

0.00 

424 

Communicate  and  Report 

i  6336 

iSOUND  LOCALIZATION  i07-Send  Message 

0.00 

1.20 

0.00 

425 

Communicate  and  Report 

i  0330 

jSOUND  LOCALIZATION  1 08-Verbal  Order 

0.00  i 

1.20 

0.00 
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B 

c 

D 

E 

F 

G 

H 

1 

1 

B 

\|^n 

anK 

Communicate  and  Report 

0330 

SOUND  LOCALIZATION 

09-Roll  Up  Reports 

0.00 

1.20 

0.00 

427 

Communicate  and  Report 

0330 

SOUND  LOCALIZATION 

10-Call  to  Conference 

0.60 

1.20 

0.00 

428 

Communicate  and  Report 

0330 

SOUND  LOCALIZATION 

26-Receive  Digital  Message 

0.00 

1.20 

0.00 

Communicate  and  Report 

0330 

SOUND  LOCALIZATION 

27-Input  data  Into  Computer 

0.00 

1.20 

0.00 

gfg»i 

Communicate  and  Report 

0330 

SOUND  LOCALIZATION 

28-Send  Digital  information 

0.00 

1.20 

0.00 

Communicate  and  Report 

0340 

CONTROL  PRECISION 

01 -Receive  and  Record/Analog 

0.00 

1.20 

0.00 

Communicate  and  Report 

0340 

CONTROL  PRECISION 

02-Pass  Information 

0.00 

1.20 

0.00 

Communicate  and  Report 

0340 

CONTROL  PRECISION 

03-Llsten-Receive  Information 

0.00 

1.20 

0.00 

Communicate  and  Report 

OMO 

CONTROL  PRECISION 

04-Secondary  Monitor 

0.00 

1.20 

0.00 

L435 

Communicate  and  Report 

0340 

CONTROL  PRECISION 

05-Log  Message 

0.00 

1.20 

0.00 

Communicate  and  Report 

0340 

CONTROL  PRECISION 

66-Route  (Outside  the  Section) 

0.00 

1.20 

0.00 

gga 

Communicate  and  Report 

0340 

CONTROL  PRECISION 

07-Send  Message 

0.00 

1.20 

0.00 

gica 

Communicate  and  Report 

0340 

CONTROL  PRECISION 

08-Verbal  Order 

0.00 

1.20 

0.00 

L439 

Communicate  and  Report 

0340 

CONTROL  pREQ|3,QN 

09-Roll  Up  Reports 

0.00 

1.20 

0.00 

Communicate  and  Report 

0^6 

CONTROL  PRECISION 

10-Call  to  Conference 

0.00 

1.20 

0.00 

441 

Communicate  and  Report 

0340 

CONTROL  PRECISION 

26-Receive  Digital  Message 

0.00 

1.20 

0.00 

442 

Communicate  and  Report 

0340 

CONTROL  PRECISION 

27-Input  data  Into  Computer 

0.00 

1.20 

0.00 

443 

Communicate  and  Report 

0340 

CONTROL  PRECISION 

28-Send  Digital  Information 

0.00 

1.20 

0.00 

EE3 

Communicate  and  Report 

0350 

RATE  CONTROL 

01 -Receive  and  Record/Analog 

0.00 

1.20 

0.00 

I  445 

Communicate  and  Report 

0350 

f^7e  control 

02-Pass  Information 

0.00 

1.20 

0.00 

E£3 

Communicate  and  Report 

L  0350 

RATE  CONTROL 

03-Listen-Receive  Information 

6.66 

1.20 

0.00 

I  447 

Communicate  and  Report 

0350 

RATE  CONTROL 

04-Secondary  Monitor 

0.00 

1.20 

0.00 

Communicate  and  Report 

0350 

RATE  CONTROL 

05-Log  Message 

6.66 

1.20 

0.00 

EE9 

Communicate  and  Report 

0350 

RATE  CONTROL 

06-Route  (Outside  the  Section) 

0.00 

1.20 

0.00 

450 

Communicate  and  Report 

0350 

IRATE  CONTROL 

07-Send  Message 

0.00 

1.20 

0.00 

451 

Communicate  and  Report 

"0356~ 

RATECONTROL . 

0.00 

1.20 

0.00 

Communicate  and  Report 

f'o^ . 

. 

69-Roil  Up  Reports  | 

6.66 

1.20 

0.00 

gl«-kl 

Communicate  and  Report 

0350  I 

E  CONTROL  J 

0.00 

1.20 

0.00 

g!gl 

Communicate  and  Report  j 

0350 

ft^TE  CON  jroL 

0.00 

1.20 

0,00 

gEa 

C^ommu^^ 

"0350 . 

RiVTE  CONTROL 

000  ; 

1,20 

0.00 

456 

Communicate  and  Report  1 

"  6350" 

RATE  CONTROL 

28-Send  Digital  Information 

6.66  : 

1.20 

0.00 

457 

Commuriicate  and  Report  i 

i  0360 

iWRIST-FINGER  SPEED 

0 1  -Receive  a nd  Rerord/Aria^  | 

0.00 

1.20 

0.00 

g!CT 

Comrnunic^te 

'  0360 

IWRIST-FINGER  SPEED 

02-Pass  inforrnation 

oCool 

1 

1.20 

0.00 

gCEl 

Communjcatea 

!  0360 

jWRJsf-FINGER  SPEED 

03-Listeri-Receiye  Inforrriatiop 

6.06 ' 

i 

1.20 

0.00 

460 

Communicate  and  Report 

i . 0360 . i 

jWRIST-FINGER  SPEED 

64-Secoridaiy^^^M^^ 

\0M . 1 

1.20 

0.00 

461 

Communicate 

1 . 

[WRIST-FINGER  speed 

65-Log  Message  j 

1  ooo' 

1.20 

0.00 

gfael 

Communic^^^ 

1  0360  1 

1  aoo  ^ 

1.20 

0.00 

gfAl 

Cornrnu nicate  and  Report  ^ 

* . . ^0^ 

07-Send  Message 

I  6^66 : 

1.20 

0.00 

Comniunk^te 

|'’''"0366'''"| 

68-Verbai  Order  j 

j  aoo 

1.20 

0.00 

465 

Cornrnu  nicat^^ 

1 . 6360 

VWIST-FINGER  SPEED 

69-Roll  Up  Reports 

1  0.06 

r 

1.20 

0.00 

466 

Communicate  and  Report 

j . 0360 . ■ 

WRIST-FINGER  SPEED 

Vo-Cail  to  Conference 

1  6,06 

i 

s_ 

1.20 

0.00 

467 

Communicate  and  Report 

i . o36o“ 

wrist-finger  speed 

26-Receive  Digital  Message 

166 

1 

1.20 

1.20 

468 

Communicate  and  Report 

1  6360 . 

WRIST-FINGER  SPEED 

27-Input  data  Into  Computer 

166“ 

1.20 

1.20 

469 

Communicate  and  Report 

1  0360 

IWRIST-FINGER  SPEED 

28-Send  Digital  Information 

1^60 

1.20 

1.20 

470 

Communicate  and  Report 

0370 

FINGER  DEXTERITY 

61 -Receive  and  Record/Analog 

1  too 

1.20 

1.20 

471 

Communicate  and  Report 

1  0370 

FINGER  DEXTERITY 

02-Pass  Information 

1  1,00 

\ 

1.20 

1.20 

472 

Communicate  and  Report 

0370 

FINGER  DEXTERITY 

i  03-Listen-Receive  Information 

i  0.00 

1.20 

0.00 

473 

Communicate  and  Report 

6370^ 

FINGER  DEXTERITY 

04-Secondary  Monitor 

\  1.6b' 

1.20 

1.20 

474 

Communicate  and  Report 

1“  6376“ 

fInGER  DEXTERITY 

05^Log 

i  too" 

1.20 

1.20 

475 

Communicate  and  Report 

i  ~03^ . 

IRNGER  DEXTERITY 

OS-Route  (Outeide  the  Section) 

1  0.60 

r" 

L 

1.20 

0.00 

476 

Communicate  and  Report 

1  0370 

FINGER  DEXTERITY 

07-Send  Message 

1  1.00 

1.20 

1.20 

477 

Communicate  and  Report 

!  0370 

i  FINGER  DEXTERITY 

68-Verbai  Order 

i 

1.20 

0.00 

478 

Communicate  and  Report 

1  0370 

IFINGER  DEXTERITY 

69-Roll  Up  Reports 

r'Tob 

i 

S 

1.20 

1.20 
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B  1 

C 

1 

Communicate  and  Report  | 

0370 

gfeW 

Communicate  and  Report  1 

0370 

gliW 

Communicate  and  Report  | 

0370 

gga 

Communicate  and  Report  | 

0370 

glAM 

Communicate  and  Report 

0380 

1  484  1 

Communicate  and  Report 

0380 

Eia 

Communicate  and  Report 

0380 

14861 

Communicate  and  Report 

0380 

g!CT 

Communicate  and  Report 

0380 

1  488  1 

Communicate  and  Report 

0380 

g!;M 

Communicate  and  Report 

0380 

1  490  1 

Communicate  and  Report 

0380 

gsi 

Communicate  and  Report 

0380 

Communicate  and  Report 

0380 

Communicate  and  Report 

0380 

gb!l 

Communicate  and  Report 

0380 

Communicate  and  Report 

0380 

g|s!J 

Communicate  and  Report  J 

0390 

gga 

Communicate  and  Report  | 

0390 

im^^i^iiii 

Communicate  and  Report  | 

0390 

gE51 

Communicate  and  Report  | 

0390 

1  500  1 

Communicate  and  Report  j 

0390 

PtiW 

Communicate  and  Report  I 

0390 

502 

Communicate  and  Report  1 

0390 

503 

Communicate  and  Report  i 

0390 

504 

Communicate  and  Report  ! 

0390 

505 

Communicate  and  Report  j 

0390 

506 

Communicate  and  Report  | 

0390 

sail 

Communicate  and  Report  j 

0390 

508 

Communicate  and  Report 

1  bbbb 

509 

Communicate  ani^^ 

1  b4ob 

isniM 

Communicate^^^ 

[  b^o 

gni 

Communicate  and^^  i 

cwbb 

lilFl 

Conrirnunicate  and  Report 

i  0400 

Communicate  and  Report 

1  0400 

lilEl 

Cornrriunicate  and  Report  i 

i  0400 

KM 

Communicate  and  Report  i 

!  0400 

KM 

Communicate  and  Report 

b4bo 

517  I 

Communicate  and  Report 

. biob" 

Communicate  and  Report 

0400 

519 

Communicate  and  Report  | 

0400 

j59![9!9H9i99*®  99^.  ^®P9F  ! 

0400“ 

521 

Communicate  and  Report  | 

0400 

Communicate  and  Report 

0410 

gyjci 

ky.iTMy  n'P®^ 

"”04lb 

gfegi 

Communicate  and  Report 

l"b4ib 

■^Wf 

Communicate  and  Report  ! 

[  0410  ” 

Communicate  and  Report 

flMib' 

g^ 

Communicate  and  Report 

1  0410 

gl^ 

Communicate  and  Report 

i  0410 

1529 

Communicate  and  Report 

r’b4?b 

Communicate  and  Report 

1  0410 

gjcgl 

Communicate  and  Report 

i  0410 

FINGERDEXTERITY^ 
"  FINGiR  DEXTC 
FINGER 


!™GERDEXT]^TY  28- 

Jb^ 

. 

DEXTERiw  ' 

. 08- 

"  j  :jo; 

Imanualdextc^^^  _ 

. .  “  2f. 

. . 

arm-handstea^^^^^  b^ 

ARM-HAND 

ARM-HAND 

ARM-HANDSf^^^ 

]aRM-HAN^^^^  Og; 

jARI^HAND^ 

Iarm^handct^ 

|arm:HANDCT  127- 

IawviSan^^^ 
jMULfmMBC^ 
jMULfiTfMB 

iMyLflilMB^^TO 

[MULfiTipBcboi^^ 

iMULfl-^ 

[MULfS^^ 

lEXfiENTFLE^lLi^ 

. 

iixfENf  F^  J  03- 

IeXTENT  FLp  J05 

j  EXTENT  FlEX^  .  ~  06- 

[Bailor  f[^  ^ 

IextentflexEI  . m 

lixfi^fFLExi  . i^ 

Te^NT  FLEXIBILITY . . . ]  1  b 


-Call  to  Conference 
-Receive  Digital  Message 
-Input  data  Into  Computer 

-Send  Digitei  information _ 

-Receive  and  Record/Anaiog 

IpMfPf'l'ion 

-Usten-Receive  Information 
-Sec^nda^^ 

-Log  Message 

-Route  (Outside  the  Section) 

-Send  Message _ 

-Verbal  Order 
-RoijJJpReporte 
-Call  to  Conferen^ 

:!npy?  JDi9. 

-Send  Digital  Information 

-Pass  Information 
-Usten-Receive  Information 

‘?99999*?,[y. 

:k99  Message 

:Send__Message_^^_ . . . . 

-Verbai  Order . . . . . 

-Roil  Up  Report _ 

:P®  lli®  _?9n!®r®o.9®-_ 

-Receive  Digite^^ 

-Input  data  Into  Computer 

:f^®9®!y®  .999 
-Pass  Informa^^ 
-Usten-Receive  i 
-Secondary  Monitor 
:k®9...M®®?®9® . 

:§®!].kM®®®?9®. 

-ye£ba[  Orde£_^_^^  . . . . 

:?9lLy  P 

:P9!!J!9  .P99f®r®09® 

-Receive  Digital  Message 

rinpy?.  k?t®  M9p9'TR4?®r 

-Send  Digital  information 
-Receive  and  Record/Anaiog 
-Pass  Information 
-Usten-Receive  Information 

:§®99n9.®JY.^M^^ 

-Log  Message 

-Route  (Outside  the  Section) 
-Send  Message 
-Verbal  Order 

^"§9LyP._l®P9T?5. 

-Cali  to  Conference 


ijxpo^i 
[  aob  [ 
YTmI 
] ibb  T 
I  abo  y 
j  ai^  i 

I . o'bdl 

'I  b^b'l 
I"  aob] 
]"aboT 

I . bxio . ! 

i  0.00  I 
0.00  I 

abb! 
. ’T!!'i3'T 

. 1.'l3"| 

y'iTibT 

i  b.bb  i 
jbbb  1 
[bbo'l 
I  b.bb  1 

faob”’| 
T'b.bbl 
i  bbb  I 
J'b.bb''] 
I  "abb"] 
b.bb"[ 
b.bo  1 
b.bb  i 
bbb i 

. bbb] 

abb} 
. abb' . I 


1^1 

Eiiia 

P51 

EESII 

Bli!il 

R!i| 
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_ 1 _ B _ 

C  1 

D 

E 

F 

1 

H 

1 

1 

1 

r  '"'f  •■■■■■■ 

^20^ 

:..petkt 

Communicate  and  Report 

0410 

EXTENT  FLEXIBILITY 

26-Receive  Digital  Message 

o.od 

0.00 

533 

Communicate  and  Report 

0410 

EXTENT  FLEXIBILITY 

27-Input  data  Into  Computer 

0.00 

1.20 

0.00 

Eg! 

Communicate  and  Report 

0410 

EXTENT  FLEXIBILITY 

28-Send  Digital  Information 

0.00 

1.20 

0.00 

Communicate  and  Report 

0420 

DYNAMIC  FLEXIBILITY 

01 -Receive  and  Record/Analog 

0.06 

1.20 

0.00 

536 

Communicate  and  Report 

0420 

DYNAMIC  FLEXIBILITY 

02-Pass  Information 

0.00 

1.20 

0.00 

537 

Communicate  and  Report 

0420 

DYNAMIC  FLEXIBILITY 

03-Listen-Receive  Information 

0.00 

1.20 

0.00 

Communicate  and  Report 

0420 

DYNAMIC  FLEXIBILITY 

04-Secondary  Monitor 

0.00 

1.20 

0.00 

I  539  I 

Communicate  and  Report 

0420 

DYNAMIC  FLEXIBILITY 

05-Log  Message 

0.00 

1.20 

0.00 

issi 

Communicate  and  Report 

0420 

DYNAMIC  FLEXIBILITY 

06-Route  (Outside  the  Section) 

0.06 

1.20 

0.00 

^41J 

Communicate  and  Report 

0420 

DYNAMIC  FLEXIBILITY 

07-Send  Message 

0.00 

1.20 

0.00 

Communicate  and  Report 

0420 

DYNAMIC  FLEXIBILITY 

OS-Verbal  Order 

0.00 

1.20 

0.00 

Igkl 

Communicate  and  Report 

0420 

DYNAMIC  FLEXIBILITY 

09-Roll  Up  Reports 

0.00 

1.20 

0.00 

Communicate  and  Report 

6426 

DYNAMIC  FLEXIBILITY 

10-Call  to  Conference 

“^6.60^ 

1.20 

0.00 

■aif 

Communicate  and  Report 

0420 

DYNAMIC  FLEXIBILITY 

26-Receive  Digital  Message 

0.00 

L 

1.20 

0.00 

Communicate  and  Report 

0420 

DYNAMIC  FLEXIBILITY 

27-Input  data  Into  Computer 

0.00 

1.20 

0.00 

547 

Communicate  and  Report 

0420 

DYNAMIC  FLEXIBILITY 

28-Send  Digital  Information 

0.00 

1.20 

0.00 

548 

Communicate  and  Report 

6436 

SPEED  OF  LIMB  MOVEMEf 

01 -Receive  and  Record/Analog 

0.00 

1.20 

0.00 

Communicate  and  Report 

0430 

SPEED  OF  LIMB  MOVEMEf 

02-Pass  Information 

0.00 

1,20 

0.00 

»W«M 

Communicate  and  Report 

0430 

SPEED  OF  LIMB  MOVEMEf 

03-Llsten-Recelve  Information 

0.00 

1.20 

0.00 

1551 

Communicate  and  Report 

0430 

SPEED  OF  LIMB  MOVEMEf  04-Secondary  Monitor 

0.00 

1.20 

0.00 

Communicate  and  Report 

0I36 

SPEED  OF  LIMB  MOVEMEf  05-Log  Message 

0.00 

1.20 

0.00 

Communicate  and  Report 

0430 

SPEED  OF  Limb  MOVEMEfoe-Route  (Outside  the  Section) 

0.00 

1.20 

0.00 

1  554 

Communicate  and  Report 

0430 

SPEED  OF  LIMB  MOVEME(07-Send  Message 

0.00 

1.20 

0.00 

Communicate  and  Report 

0430 

SPEED  OF  LIMB  MOVEME^OS-Verbal  Order 

0.00 

1.20 

0.00 

Communicate  and  Report 

6436 

SPKD  OF  LIMB  MOVEMEi 09-Roll  Up  Reports 

0.00 

1.20 

0.00 

557 

Communicate  and  Report 

0430 

SPEED  OF  LIMB  MOVEMEf!  10-Call  to  Conference 

i  : 

1.20 

0.00 

558 

Cprrimunicate  and  Report 

0430 

SPEED  OF  LIMB  MOVEMEf 

26-Receive  Digital  Message 

1  ^00” 

1.20 

0.00 

E3tl 

Cornrnurii(^te^a^^ 

0430 

SPEED  OF  LIMB  MOVEMEf  27-Input  data  Into  Computer 

faop! 

1.20 

0.00 

Em!! 

0430 

SPEED  OF  LIMB  MOVEMEf 

28-Send  Digital  Information 

0.00 

1.20 

0.00 

^61 

an 

[0^ . 

GROSS  BODY  EQUILIBRIMOI-Receive  and  Record/Analog 

0.00 

1.20 

0.00 

Communicate  and  Report  i 

0440  i 

C5ROSS  BODY  EQUILIBRIlJ  02-Pass  Information 

oxio 

r 

1.20 

0.00 

1  563 

Communicate  and  Report 

0440  I 

GROSS  BODY  EQUiUBRIU!03-Listen-Receive  information 

aoo 

1.20 

0.00 

Comrnunicate  and  Report  ] 

i  0440  1 

GROSS  BODY  EQUILIBRIA  04-Secondary  Monitor 

d.oo 

1.20 

0.00 

Communicate  and  Repprt  j 

f'''''04io1 

GROSS  BOTY  EQUrUBRIUiO^Log  Message 

dloF 

r 

1.20 

0.00 

Communicate  and  Report 

. ouo] 

GROSS  BODY  EQUILIBRIU 

06-Route  (Outside  the  Section) 

Foo 

L 

1.20 

0.00 

Elitifl 

Communicate  and  Report 

1  0440  \ 

GROSS  BODY  EQUILIBRIU 

07-Send  Message 

0.00 

L 

1.20 

0.00 

1  568 

Communicaie  and  Report 

[0440 1 

GROSS  BODY  EQUILIBRIU!  08-Verbal  Order 

6^0^ 

I 

1.20 

0.00 

■aw 

Communicate  and  Reoort  i 

r  0440  ] 

gross  body  EQUiLlBRiOoi-Roil  Up  Reports 

add 

1.20 

0.00 

■arw 

Communicate  and  Report  i 

\"0440  1 

GROSS  BODY  EQUiUBRiUlO-Cali  to  Conference 

add 

1.20 

0.00 

1  571 

Communicate  and  Report 

I  “  6440 

GROS~S~BdDY  EQUIUBRILjli^i^Receive  Digital  Message 

[add 

1.20 

0.00 

Communicate  and  Report 

1  b440 i 

IGROSS  BODY  EQUILIBRIU27-lnput  data  Into  Computer 

I  add 

1.20 

0,00 

573 

Communicaie  and  Repon  | 

0440 

IGROSS  BODY  EQUILIBRIU;28-Send  Digital  Information 

0.00  1 

1.20 

0.00 

574 

Communicate  and  Report 

0450 

GROSS  BODY  COORDINA' 

01-Receive  and  Record/Analog 

0.00 

1.20 

0.00 

575 

Communicate  and  Report 

0450 

GROSS  BODY  COORDINA' 

02-Pass  Information 

0.00 

1.20 

0.00 

EHii 

Communicate  and  Report 

0450" 

GROSS  BODY  COORDINA' 

03-Listen-Receive  Information 

0.00 

1.20 

0.00 

Communicate  and  Report 

0450 

GROSS  BODY  COORDINA' 

04-Secondary  Monitor 

o.od 

1.20 

0.00 

Communicate  and  Report 

i  0450" 

GROSS  BODY  COORDINA' 

05-Log  Message 

i  0.00 

L 

1.20 

0.00 

Communicate  and  Report 

>  0450 

GROSS  BODY  COORDINA' 

06-Route  (Outside  the  Section) 

1.00 

1.20 

1.20 

I  580 

Communicate  and  Report 

r  0450 

GROSS  BODY  COORDINA' 

07-Send  Message 

0.00 

1.20 

0.00 

Egl 

Communicate  and  Report 

[0450 

GROSS  BODY  COORDINA' 

08-Verbal  Order 

0.00 

1.20 

0.00 

582 

Communicate  and  Report 

1 . 6450" 

GROSS  BODY  COORDINA' 

09-Roll  Up  Reports 

0.00 

1 

1.20 

0.00 

583 

Communicate  and  Report 

1  0450 

GROSS  BODY  COORDINA 

10-Call  to  Conference 

0.00 

1.20 

0.00 

584 

Communicate  and  Report 

[  04^  " 

IGROSS  BODY  COORDINA 

26-Receive  Digital  Message 

0.00 

1.20 

0.00 
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1  B  1 

C 

D 

E 

F 

G 

H 

1 

1 

J^OQIFB 

1 

■  ’Acll- 

Communicate  and  Report 

0450 

GROSS  BODY  COORDINAl 

27-Input  data  Into  Computer 

0.00 

1.20 

0.00 

586 

Communicate  and  Report 

0450 

GROSS  BODY  COORDINA' 

28-Send  Digital  Information 

0,00 

1.20 

0.00 

587 

Communicate  and  Report 

0460 

STATIC  STRENGTH 

01 -Receive  and  Record/Analog 

0.00 

1.20 

0.00 

Communicate  and  Report 

0460 

STATIC  STRENGTH 

02-Pass  Information 

0.00 

1,20 

0.00 

589 

Communicate  and  Report 

0460 

STATIC  STRENGTH 

03-Llsten-Receive  Information 

0.00 

1.20 

0.00 

590 

Communicate  and  Report 

0460 

STATIC  STRENGTH 

04-Secondary  Monitor 

0.00 

1.20 

0.00 

591 

Communicate  and  Report 

0460_j 

STATIC  STRENGTH 

05-Log  Message 

0.00 

1.20 

0.00 

Communicate  and  Report 

0460 

^ficsfRENGTH 

06-Route  (Outside  the  Section) 

0.00 

1.20 

0.00 

■a?i« 

Communicate  and  Report 

0460 

STATIC  STRENGTH 

07-Send  Message 

0.06 

1.20 

0.00 

Communicate  and  Report 

0460 

STATIC  STRENGTH 

08-Verbal  Order 

0.00 

1.20 

0.00 

Communicate  and  Report 

0460 

STATIC  STRENGTH 

09-Roll  Up  Reports 

0-.0.0. 

1.20 

0.00 

596 

Communicate  and  Report 

0460 

STATIC  STRENGTH 

1 0-Call  to  Conference 

0.00 

1.20 

0.00 

mM 

Communicate  and  Report 

6460 

STATIC  STrInGT  H 

26-Receive  Digital  Message 

0.00 

1.20 

0.00 

598 

Communicate  and  Report 

0460 

STATIC  STRENGTH 

27-Input  data  Into  Computer 

0.00 

1.20 

0.00 

599 

Communicate  and  Report 

0460 

STATIC  STRENGTH 

28-Send  Digital  Information 

0.00 

1.20 

0.00 

600 

Communicate  and  Report 

0470 

EXPLOSIVE  STRENGTH 

01-Receive  and  Record/Analog 

0.00 

1.20 

0.00 

601 

Communicate  and  Report 

0470 

EXPLOSIVE  STRENGTH 

02-Pass  Information 

0.00 

1.20 

0.00 

602 

Communicate  and  Report 

0470 

EXPLOSIVE  STRENGTH 

03-Llsten-Receive  Information 

0.00 

1.20 

0.00 

iniici 

Communicate  and  Report 

0470 

EXPLOSIVE  STRENGTH 

04-Secondary  Monitor 

0.00 

1.20 

0.00 

I  604 

Communicate  and  Report 

0470 

EXPLOSIVE  STRENGTH 

05-Log  Message 

0.00 

1.20 

0.00 

ES9 

Communicate  and  Report 

.._p47q_. 

EXPLOSIVE  STRENGTH 

06-Route  (Outside  the  Section) 

6.66 

1.20 

0.00 

I  606 

Communicate  and  Report 

0470 

EXPLOSIVE  STRENGTH 

07-Send  Message 

0.00 

1.20 

0.00 

^3 

Communicate  and  Report 

0470 

1  EXPLOSIVE  STRENGTH 

08-Verbal  Order 

0.00 

1.20 

0.00 

11^3 

Communicate  and  Report 

0470 

B^LOsiv^  strength 

09-Roll  Up  Reports 

0.00 

1.20 

0.00 

609 

Communicate  and  Report 

0470 

1  EXPLOSIVE  STRENGTH 

10-Call  to  Conference 

0.00 

1.20 

0.00 

610 

Communicate  and  Report 

1  0470 

EXPLOSIVE 

26-Receive  Digital  Message 

laooi 

1.20 

0.00 

611 

. 0470  ' 

lEXPLOSiVE  STRENGTH 

27:|nput  date  Into  j 

j'aoo^ 

1.20 

0.00 

612 

Communicate  and  Report 

647Ti“ 

i  EXPLOSIVE  STRENGTH 

1 28-Send  Digital  Information  ! 

loMI 

1.20 

0.00 

gflcl 

Comrnunic^te  and 

6480 

DYNAMIC  STRENGTH 

[01 -j^ceiye  and  ^cor^  j 

[  6^60  i 

1.20 

0.00 

614 

Communicate  and  Report 

1  6480 

. 1 

!62-Pass  information  | 

\0M 

1.20 

0.00 

Communicate  and  Report 

i  6480 

DYNAMIC  STRENGTH 

03-Listen-Receive  Information  j 

0.00 

1.20 

0.00 

616 

Cornmunlca^^^^^ 

1  0480 

DYNAMIC  STO 

64-Secor^da^  I 

OJOO 

1.20 

0.00 

617 

Comrnuni^^^^ 

1 . 0486^'''i 

!  DYNAMiC  3f RENQf H 

OS-Log  Message 

6.60^ 

1.20 

0.00 

linsi 

Communicate  and  Report  j 

1 . MSO . i 

IDYNAMICCT 

Oe-Route  (Outsidj^^to 

1.20 

0.00 

■siM 

Cornrnunicate  and  Report  i 

1 . 0480 

IDYIMAMIC  STRENGTH  ^  " 

07-Send  Message 

6.00 

1.20 

0.00 

620 

Cornmunicate^  a^^^^ 

i . 0480 

1  DYNAMIC  STRENGTH . 

OS-Verbal  Order 

0.00 

1.20 

0.00 

621 

tonmunicate  and  import  J 

1 . 04S0 

;  DYNAMIC  STRENG^ . 

Og-Rofl  Up  Reports 

aoo 

r 

1.20 

0.00 

Cornmunte^ 

1  “6480^  ' 

[dynamic  STRB^  . 

.9991919599 

aoo' 

p 

L- 

1.20 

0.00 

ggcl 

Communicate  and  Report 

I  0480 

[DYNAMIC  STRENGfH 

26-Receive  Digital  Message 

'  0.66  \ 

L 

1.20 

0.00 

1  624 

Communicate  and  Report 

1 . 0480  ' 

!  DYNAMIC  STRENGTH ^ 

27-Input  data  Into  Conriputer 

6.66 

L 

1.20 

0.00 

Communicate  and  Report  1 

i  6480 

[DYNAMIC  STRENGTH 

28-Send  Digital  Information 

6.60 

1 

1.20 

0.00 

1  626 

Communicate  and  Report  j 

0490  iTRUNK  STRENGTH 

01-Receive  and  Record/Analog 

1  6.00 

1 

1.20 

0.00 

0490 

TRUNK  STRENGTH 

02-Pass  Information 

0.00 

r~ 

J _ 

1.20 

0.00 

628 

Communicate  and  Report 

0490 

TRUNK  STRENGTH 

OS-Listen-Receive  Information 

6.66 ! 

1.20 

0.00 

629 

Communicate  and  Report  i 

0490 

TRUNK  STRENGTH 

64-Secondary  Monitor 

a66 . 

~ 

1 

1.20 

0.00 

630 

Communicate  and  Report  I 

0490 

ITRUNK  STRENGTH 

05-Log  Message 

0.60 

! 

1.20 

0.00 

631 

Communicate  and  Report 

0490 

TRUNK  STRENGTH 

06-Route  (Outside  the  Section) 

r6“6o' 

1.20 

0.00 

Communicate  and  Report 

0490 

TRUNK  STRENGTH 

07-Send  Message 

1  6766 

1.20 

0.00 

gggi 

Comrnunicate  and  Report 

0490 

iTRUNK  STRENGTH 

08-Verbal  Order 

j  6.06 

i 

1.20 

0,00 

1  634 

Communicate  and  Report 

0490 

ITRUNK  STRENGTH 

09-Roll  Up  Reports 

1 0.00 

\ 

1.20 

0.00 

Communicate  and  Report 

0490  iTRUNK  STRENGTH 

i  o-Caii  to  Conference 

i  b'.do 

1.20 

0.00 

636 

Communicate  and  Report 

0490 

iTRUNK  STRENGTH 

26-Receive  Digital  Message 

raob’ 

1.20 

0.00 

637 

Communicate  and  Report 

6490 

[TRUNK  STRENGTH 

27-lnput  data  Into  Computer 

0.00  5 

1.20 

0.00 
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_ B _ 1 

C 

D 

E  1  F 

1 

H 

1 

ft 

Communicate  and  Report 

0490 

TRUNK  STRENGTH 

28-Send  Digital  Information 

0.00 

1.20 

639 

Communicate  and  Report 

0500 

STAMINA 

01 -Receive  and  Record/Analog 

0.00 

1.20 

640 

Communicate  and  Report 

0500 

STAMINA 

02-Pass  Information 

0.06  J 

1.20 

Communicate  and  Report 

0500 

STAMINA 

03-Listen-Receive  Information 

0.00 

1.20 

gggi 

Communicate  and  Report 

dsoo 

STAMINA 

04-Secondary  Monitor 

0.00 

1.20 

Communicate  and  Report 

0500 

STAMINA 

05-Log  Message 

0.00 

1.20 

iggi 

Communicate  and  Report 

0500 

STAMINA 

06-Route  (Outside  the  Section) 

.9M.J 

1.20 

Communicate  and  Report 

0500 

STAMINA 

07-Send  Message 

0.00 

1.20 

ggni 

Communicate  and  Report 

0500 

STAMINA 

08-Verbal  Order 

0.00 

1.20 

liggi 

Communicate  and  Report 

0500 

STAMINA 

09-Roll  Up  Reports 

0.00 

1.20 

648 

Communicate  and  Report 

0500  jSTAMINA 

10-Call  to  Conference 

0.00 

1.20 

649 

Communicate  and  Report 

0500 

STAMINA 

26-Receive  Digital  Message 

b.oo 

1.20 

650 

Communicate  and  Report 

0500 

STAMINA 

27-Input  data  Into  Computer 

0,00 

1.20 

651 

Communicate  and  Report 

0500 

STAMINA 

28-Send  Digital  Information 

0.00 

1.20 

Decide  and  Recommend  /  E 

0010 

ORAL  COMPREHENSION 

1 1 -Decide  Action 

0.00 

0.80 

Decide  and  Recommend  /  C 

0010 

ORAL  COMPREHENSION 

12-Decide 

1.92 

0.80 

Decide  and  Recommend  /  C 

0010 

ORAL  COMPREHENSION 

13-Recommend  Action 

1.92 

0.80 

655 

Decide  and  Recommend  /  u 
Decide  and  Recommend  /  C; 

0020 

WRITTEN  COMPREHENSIC 

1 1 -Decide  Action 

1.61 

0.80 

6620 

WRITTEN  COMPREHENSK 

12-Decide 

1.61 

0.80 

■aar^ 

Decide  and  Recommend  /  C 

0020 

WRiffEN  COMPREHENSli 

1 3-Recommend  Action 

1.61 

0.80 

Decide  and  Recommend  /  C 

0030 

ORAL  EXPRESSION 

11 -Decide  Action 

0.00 

0.80 

659 

Decide  and  Recommend  /  C 

0030 

ORAL  EXPRESSION 

12-Decide 

2.34 

0.80 

660 

Decide  and  Recommend  /  E 

0030 

ORAL  ixPRESSION 

13-Recommend  Action 

2.34 

0.80 

^1 

Decide  and  Recommend  /  E 

0040 

WRITTEN  EXPRESSION 

11 -Decide  Action 

0.00 

0.80 

662 

Decide  and  Recommend  /  C 

0040 

WRITTEN  EXPRESSION 

12-Decide 

0.00 

0.80 

Decide  and  Recommend  /  E 

0040 

WRITTEN  EXPRESSION 

1 3-Recommend  Action 

0.00 

0.80 

g!gl 

Decide  and  Recommend  /  E 

0050 

MEMORIZATION 

11 -Decide  Action 

2.61 

0.80 

665 

Decide  and  Recommend  /  E 

0050 

MEMORIZATION 

12-Decide 

2.61 

0.80 

666 

Decide  and  R^ 

0050 

MEMORIZATION 

13-Recommend  Action 

Z61 

0.80 

667 

Decide  and  Recom^^ 

0060 

PROBLEM  SENSITIVITY 

11 -Decide  Action 

2.12  : 

M 

0.80 

Decide  and  Recommend  /  C 

dded 

PROBLEM  SENSif  iviTY 

12-becide 

2.12  1 

0.80 

669 

ciecide  and  Recommend  /  C 

0060 

PROBLEM  SENSITIVITY 

1  ^Rec^mmer^  ^ion  ^ J 

2.12  ; 

0.80 

670 

Decide  and  Rec^  /  E 

0070 

ORIGINALITY 

J  J  ■P®®id® 

aoo"* 

0.80 

671 

Decide  a^  /  C 

0070 

ORIGINALITY 

jf^Decide^^^ . . . . . : 

6.66 

0.80 

Decide  and  Recommend  /  C 

0070 

ORIGINALITY 

j  3-Recommend  ^ 

1  0.00 

0.80 

1  673 

Decide  and  Reconrimend  /  C 

0080 

FLUENCY  OF  IDEAS 

11 -Decide  Action 

i'0.06' 

1 

0.80 

Decide  and  Recommend  /  E 

0080 
^  doeo  " 

FLUENCY  OF  IDEAS 

12-Decicle 

i'6.06 

r 

0.80 

675 

Decide  and  Recommend  /  E 

IFLUENCY  OF  IDEAS 

Action 

1  0.66 

L 

0.80 

676 

Decide  and  Recommend  /  E 

0090 

FLEXIBILITY  OF  CLOSURE 

11 -Decide  Action 

1 . T.51 . 

1 

0.80 

gga 

®  f: 

0090 

1  FLEXIBILITY  OF  CLOSURE 

12-Decide 

1 

0.80 

ggii 

Decide  and  Recommend  /  E 

i  0090 

iFLEXiBllWoFCW^ 

;  13-Recommend  Action 

\  0.00 

0.80 

679 

Decide  and  Recommend  /  E 

1^^6106“^ 

. 

11 -Decide  Action 

j  150 

0.80 

680 

Decide  and  Recommend  /  E 

i  oidd^^ 

[SBJECf  IVE  ATT ENTioisi  . . 

12-Decide 

[  1.50 

.5 _ 

0.80 

681 

Decide  and  Recommend  /  E 

1  6106 

[selective  attention 

13-Recommencl  Action 

j  o7o  o 

0.80 

lOISli 

Decide  and  Recommend  /  E 

1 . 0110“ 

SPATIAL  ORIENTATION 

1  l-Decide  Action 

1  0.66 

f _ 

0.80 

Decide  and  Recommend  /  E 

i  0110 

SPATIAL  ORIENTATION 

12-Decide 

i  0.00 

0.80 

684 

Decide  and  Recommend  /  E 

i  0110 

SPATIAL  ORIENTATION 

13-Recomr^end  Action 

!  0.00 

1 

0.80 

685 

Decide  and  Recommend  /  E 

I  0120 

VISUALIZATION 

11 -Decide  Action 

i  2.17 

0.80 

Decide  and  Recommend  /  E 

1  0120 

VISUALIZATION 

12-Decide 

i  2.17 

i 

0.80 

1  687 

Decide  and  Recommend  /  E 

1  0120 

(VISUALIZATION 

13-Recommend  Action 

j  0.00 

0.80 

1 1^1^91 

Decide  and  Recommend  /  E 

j  0130 

1  INDUCTIVE  REASONING 

11 -Decide  Action 

1  2.43 

0.80 

689 

Decide  and  Recommend  /  E 

f  0130^" 

1  INDUCTIVE 

12-Decide 

[243' 

0.80 

690 

Decide  and  Recommend  /  E 

»  0130 

IinductTve  reasoning 

1 3-Recommend  Action 

iaoo^ 

0.80 
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B 

C 

D 

E 

F 

i 

H 

1  1 

1 

K 

H 

691 

Decide  and  Recommend  /  [ 

0140 

CATEGORY  FLEXIBILITY 

1 1 -Decide  Action 

0.00 

0.80 

0.00 

692 

Decide  and  Recommend  /  H 

0140 

CATEGORY  FLEXIBILITY 

12-Decide 

0.00 

0.80 

0.00 

Decide  and  Recommend  /  C 

0140 

CATEGORY  FLEXIBILITY 

1 3-Recommend  Action 

0.00 

0.80 

0.00 

694 

Decide  and  Recommend  /  C 

0150 

DEDUCTIVE  REASONING 

1 1 -Decide  Action 

Z79 

0.80 

2.23 

695 

Decide  and  Recommend  /  C 

0150 

DEDUCTIVE  REASONING 

12-Declde 

2.79 

0.80 

2.23 

696 

Decide  and  Recommend  /  C 

0150 

DEDUCTIVE  REASONING 

1 3-Recommend  Action 

2.79 

0.80 

2.23 

697 

Decide  and  Recommend  /  C 

0160  J 

INFORMATION  ORDERING 

11 -Decide  Action 

2.85 

0.80 

2.28 

698 

Decide  and  Recommend  /  C 

0160 

INFORMATION  ORDERING 

12-Declde 

2.85 

0.80 

2.28 

699 

Decide  and  Recommend  /  C 

0160 

iNFORM^riON  ORDERING 

1 3-Recommend  Action 

2.85 

0.80 

2.28 

700 

Decide  and  Recommend  /  E 

0170 

MATHEMATICAL  REASONI 

11 -Decide  Action 

2.13 

0.80 

1.70 

701 

Decide  and  Recommend  /  C 

0170 

! MATHEMATICAL  REASONI 

12-Decide 

2.13 

0.80 

1.70 

Decide  and  Recommend  /  C 

0170 

MATHEMATICAL  REASONI 

1 3-Recommend  Action 

2.13 

0.80 

1.70 

1  703 

Decide  and  Recommend  /  C 

0180 

NUMBER  FACILITY 

11 -Decide  Action 

0.00 

0.80 

0.00 

Decide  and  Recommend  /  C 

0180 

NUMBER  FACILITY 

12-Decide 

0.00 

0.80 

0.00 

L705 

Decide  and  Recommend  /  C 

0180 

NUMBER  FACILITY 

1 3-Recommend  Action 

0.66 

0.80 

0.00 

E&3 

Decide  and  Recommend  /  C 

0190 

TIME  SHARING 

11 -Decide  Action 

2.00 

0.80 

1.60 

Decide  and  Recommend  /  C 

0190 

TIME  SHARING 

12-Decide 

2.00 

0.80 

1.60 

708 

Decide  and  Recommend  /  C 

0190 

TIME  SHARING 

1 3-Recommend  Action 

2.00 

0.80 

1.60 

Decide  and  Recommend  /  C 

0200 

SPEED  OF  CLOSURE 

11 -Decide  Action 

2.19 

0.80 

1.75 

HEI 

Decide  and  Recommend  /  C 

0200 

SPEED  OF  CLOSURE 

12-Declde 

2.19 

0.80 

1.75 

ED 

Decide  and  Recommend  /  C 

0200 

SPEED  OF  CLOSURE 

1 3-Recommend  Action 

2.19 

0.80 

1.75 

gZKM 

Decide  and  Recommend  /  C 

0210 

PERCEPTUAL  SPEED  AND 

11 -Decide  Action 

2.35 

0.80 

1.88 

Decide  and  Recommend  /  E 

0210 

PERCEPTUAL  SPEio  AND 

12-Decide 

2.35 

0.80 

1.88 

Decide  and  Recommend  /  C 

0210 

PERCEPTUAL  SPEED  AND 

1 3-Recommend  Action 

0.00 

0.80 

0.00 

1  715 

Decide  and  Recommend  / 1; 

0220 

REACTION  TIME 

11 -Decide  Action 

0.00 

0.80 

0.00 

mnm 

Decide  and  Recommend  /  [ 

6226 

REACTION  TIME 

12-Decide 

0.00 

0.80 

0.00 

UK 

Decide  and  Recommend  /  C 

0220 

R^CflONTIME 

1 3-Recommend  Action 

0.06 

0.80 

0.00 

Decide  and  Recommend  /  E 

0230 

CHOICE  REACTION  TIME 

11 -Decide  Action 

0.00 

0.80 

0.00 

EEl 

Decide  and  Recommend  /  E 

0230 

CHOICE  REACTION  TIME 

1l2-pecide 

0.00 

0.80 

0.00 

Decide  and  Recommend  /  E) 

0230 

CHOICE  REACTION  TIME 

1 3-Recommend  Action 

[*  0.00  i 

0.80 

0.00 

721 

Decide  and  ^ 

oSio  i 

iNiAR^SION* . 

1 1 -Decide  Action 

1  0.00  I 

0.80 

0.00 

722 

Decide  and  Recommend  /  lj 

I 0240 I 

1  NEAR  VISION 

12-Dedcie^^ . . . . . . . . . . : 

0.00  1 

0.80 

0.00 

ESI 

Decide  and  Rec»m^^ 

j . 0240  ^ 

INEAR  VISION 

13-Recomme^ 

[6.60* 

0.80 

0.00 

724 

D^ecide  and  ^ 

! . 0250  ^ 

1  FAR  VISION 

11 -Decide  Action 

joSo'l 

0.80 

0.00 

725 

Decide  and  Recommend  /  C 

!'0250 

FAR  VISION 

12-pecide 

[0.00  1 

0.80 

0.00 

Decide  and  Recommend  /  C 

"0250 

i  FAR  VISION 

!  o'.o6’''! 

0.80 

0.00 

WFHk 

Decide  and  Recommend  /  C 

0260  I 

INIGHTVISrON 

Action 

[  a66  1 

0.80 

0.00 

\  728 

Decide  and  Recommend  /  [j 

0260 

snTgHT  VISION 

12-Decide 

j  0.60 

0.80 

0.00 

02^“i 

i  NIGHT  visiON^ 

1 3-iRecommend  Action 

r  oToo 

0.80 

0.00 

Decide  and  Recommend  /  i 

0270 

IVISUAL  COLOR  DISCRIMIh 

1 1 -Decide  Action 

. aoo . 1 

1 

0.80 

0.00 

I  731 

0270 

VISUAL  COLOR  DISCRIMlH  12-Decide 

I . abo"! 

0.80 

0.00 

j  0270  ! 

VISUAL  COLOR  DISCRIMIh 

1 3-Recommend  Action 

1  o.bb 

0.80 

0.00 

733 

Decide  and  Recommend  /  El 

I  0280  I 

PERIPHERAL  VISION 

1 1 -Decide  Action 

I  b.bb 

0.80 

0.00 

734 

Decide  and  Recommend  /  C 

0:^ 

PERIPHERAL  VISION 

12-Decide 

i  0.00 1 

0.80 

0.00 

Kca 

Decide  and  Recommend  /  E] 

!  0280 

PERIPHERAL  VISION 

1 3-Recommend  Action 

[  6j6o  1 

0.80 

0.00 

i 0290  i 

DEPTH  PERCEPTION 

11 -Decide  Action 

|“6.o6  i 

0.80 

0.00 

I  737 

Decide  and  Recommend  /  ^ 

i  02^  "i 

DEPTH  PERCEPTION 

12-Decide 

raocTl 

0.80 

0.00 

ESI 

Decide  and  Recommend  /  d 

i  0290*  “i 

DEPTH  PERCEPTION 

1 3-Recommend  Action 

1  0.00  1 

0.80 

0.00 

Decide  and  Recommend  /  d 

I  0300" 

GLARE  SENSITIVITY 

1 1 -Decide  Action 

1  oToo 

0.80 

0.00 

740 

Decide  and  Recommend  /  d 

I  0300 ~ 

GLARE  SENSITIVITY 

12-Decide 

i  aoo  1 

L 

0.80 

0.00 

741 

Decide  and  Recommend  /  0 

0300 

GLARE  SENSITIVITY 

1 3-Recommend  Action 

rojoo : 

0.80 

0.00 

742 

Decide  and  Recommend  /  D 

I  0310  * 

GENERAL  HEARING 

11 -Decide  Action 

0.00 ; 

0.80 

0.00 

743 

Decide  and  Recommend  /  d 

j  0310  * 

GENERAL  HEARING 

12-Decide 

0.00  i 

0.80 

0.00 
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B 

D 

E 

F 

1 

H 

1 

1 

fere 

1 

•ifaSr 

kem 

Decide  and  Recommend  /  C 

0310 

GENERAL  HEARING 

1 3-Recommend  Action 

0.00 

0.80 

0.00 

[7^ 

Decide  and  Recommend  /  C 

0320 

AUDITORY  ATTENTION 

1 1-Decide  Action 

0.00 

0,80 

0.00 

Decide  and  Recommend  /  d 

0320 

AUDITORY  ATTENTION 

12-Decide 

ii7 

0.80 

1.74 

Ea 

Decide  and  Recommend  /  C 

0320 

AUDITORY  ATTENTION 

1 3-Recommend  Action 

Z17 

0.80 

1.74 

748 

Decide  and  Recommend  /  C 

0330 

SOUND  LOCALIZATION 

1 1-Decide  Action 

0.00 

0.80 

0.00 

749 

Decide  and  Recommend  /  C 

0330 

SOUND  LOCALIZATION 

12-Decide 

0.00 

0.80 

0.00 

Decide  and  Recommend  /  L 

0330 

SOUND  LOCALIZATION 

1 3-Recommend  Action 

0.00 

0.80 

0.00 

Iriai 

Decide  and  Recommend  /  D 

0340 

CONTROL  PRECISION 

11 -Decide  Action 

0.00 

0.80 

0.00 

752 

Decide  and  Recommend  /  d 

034qj 

CONTROL  PRECISION 

12-Decide 

0.00 

0.80 

0.00 

753 

Decide  and  Recommend  /  d 

0340 

CONTROL  PRECISION 

1 3-Recommend  Action 

0.00 

0.80 

0.00 

754 

Decide  and  Recommend  /  C 

0350 

RATE  CONTROL 

1 1-Decide  Action 

0.00 

0.80 

0.00 

Decide  and  Recommend  /  d 

0350 

RATE  CONTROL 

12-Decide 

0.00 

0.80 

0.00 

Decide  and  Recommend  /  C 

0350 

RATE  CONTROL 

1 3-Recommencl  Action 

0-00  , 

0.80 

0.00 

I  757 

Decide  and  Recommend  /  Cj 

0360 

WRIST-FINGER  SPEED 

1 1-Decide  Action 

0.00 

0.80 

0.00 

Decide  and  Recommend  /  d 

0360 

WRIST-FINGER  SPEED 

12-Decide 

0.00 

0.80 

0.00 

759 

Decide  and  Recommend  /  C 

0360 

WRIST-FINGER  SPEED 

1 3-Recommend  Action 

0.00 

0.80 

0.00 

760 

Decide  and  Recommend  /  E 

0370 

1 1-Decide  Action 

0.00 

0.80 

0.00 

761 

Decide  and  Recommend  /  D 

0370 

FINGER  DEXTERITY 

12-Decide 

0.00 

0.80 

0.00 

762 

Decide  and  Recommend  /  C 

0370 

FINGER  DEXTERITY 

1 3-Recommend  Action 

o.ob 

0.80 

0.00 

Kiel 

Decide  and  Recommend  /  d 

0380 

MANUAL  DEXTERITY 

1 1-Decide  Action 

0.00 

0.80 

0.00 

Decide  and  Recommend  /  0 

0380 

MANUAL  DEXTERITY 

12-Decide 

o.ob 

0.80 

0.00 

Decide  and  Recommend  /  C 

0380 

MAHUaL 

1 3-Recommend  Action 

0.00 

0.80 

0.00 

Eia 

Decide  and  Recommend  /  C 

0390 

ARM-HAND  STEADINESS 

1 1-Decide  Action 

b.bb 

0.80 

0.00 

I  767 

Decide  and  Recommend  /  C 

0390 

ARM-H^D  READINESS 

12-Decide 

0.00 

0.80 

0.00 

0390 

lARM-HAND  STEADINESS 

.l?'!?®®9iniD®^^^^  Action... 

0.00 

0.80 

0.00 

769 

Decide  and  Recommend  /  d 

0400 

iMULfi-UMBC^ 

11 -Decide  Action 

0.00  1 

0.80 

0.00 

TOil 

Decide  and  Recommend  /  C| 

0400 

|MULfl-[lMB  COORDINATIC 

12-Decide 

CM30  ; 

0.80 

0.00 

[W 

Decide  and  R^^  /  cj 

0400 

MULfi-LIMB  COORDINATIC 

1 3-Recommend  Action 

o^bb  i 

0.80 

0.00 

mSlM 

0410 

E)aENf  l=^LEXIBILITY 

1 1-Decide  Action 

b.  b  o ; 

0.80 

0.00 

773 

Decide  and  Recommend  /  d 

0410  i 

EXTENT  FLEXIBILI^^^^ 

1 12-Decide 

1  o"bb"| 

0.80 

0.00 

774 

. 0410 1 

EXTENT  FLEXIBILITY . 

1 3-Recommend  Action 

0.00  ; 

0.80 

0.00 

775 

Decide  an^^ 

0420  1 

DYNAMIC  "flexibility 

1 1-Decide  Action 

bob" 

0.80 

0.00 

776 

Decide  and  ReOT 

0420 . j 

I^Decide^ _ _ _ _ _ _ 

0.80 

0.00 

Decide  and  Recommend  /  d 

04^1 

^D^MIC  pLExiBiLjjY  ^ " 

1 3-Recommend  Action  I 

'ojob^ 

0.80 

0.00 

I  778 

P®9i^.®..®  !?®590E).®M  /.  S 

0480 

SPEED  OF  LIMB  MOVEME?  1 1-Decide  Action 

'aob  i 

L 

0.80 

0.00 

Decide  a nd  Recom mend  /  d 

0430^ 

SPEED  OF  LIMB  MOVEMEIj  12-Decide 

1 

0.80 

0.00 

Decide  and  Rerommen^d  /  p 

SPEED  OF  LIMB  MOVEMEil  1 3-Recommend  Action 

0.80 

0.00 

Eitl 

Decide  and  Recommend  /  0 

0440 

GROSS  BODY  EQUILIBRIU 

1 1-Decide  Action 

b.oo 

0.80 

0.00 

782 

Decide  and  Recommend  /  D 

0440 . 

GROSS  BODY  EQUILIBRIU  12-Declde 

abb 

0.80 

0.00 

783 

Decide  and  Recommend  /  E 

0440 

jGRlOSS  BODY  EQUILIBRIU  13-Recommend  Action 

bob 

L 

0.80 

0.00 

mm 

Decide  and  Recommend  /  p 

0450 

GROSS  BODY  COORDINaIi  1-Decide  Action 

b.bb 

0.80 

0.00 

I  785 

Decide  and  Recommend  /  p 

. 0450“ 

GROSS  BODY  COORDINA' 

12-Decide 

0.00 

0.80 

0.00 

Decide  and  Recommend  /  3 

. 0450  “ 

GROSS  BODY  COORDINA' 

1 3-Recommend  Action 

b^b 

0.80 

0.00 

787 

Decide  and  Recommend  /  p 

0460 

STATIC  STRENGTH 

1 1-Decide  Action 

0.00 

0.80 

0.00 

788 

Decide  and  Recommend  /  p 

[  04^  " 

STATIC  STRENGTH 

12-Declde 

0.00 

0.80 

0.00 

789 

Decide  and  Recommend  /  P 

0460 

STATIC  STRENGTH 

1 3-Recommend  Action 

1  o.ob 

0.80 

0.00 

790 

Decide  and  Recommend  /  3 

0470 

EXPLOSIVE  STRENGTH 

1 1-Decide  Action 

1  0.00 

0.80 

0.00 

ESI 

Decide  and  Recommend  /  P 

1 0470  * 

EXPLOSIVE  STRENGTH 

12-Decide 

0.00 

! 

0.80 

0.00 

I  792 

Decide  and  Recommend  /  P 

r”  b470  " 

EXPLOSIVE  STRENGTH 

1 3-Recommend  Action 

0.00 

L 

0.80 

0.00 

Decide  and  Recommend  /  P 

1  0480 

I  DYNAMIC  STRENGTH 

11 1-Decide  Action 

0.00 

0.80 

0.00 

Egl 

Decide  and  Recommend  /  3 

i  0480 

DYNAMIC  STRENGTH 

12-Decide 

o.ob 

\ 

0.80 

0.00 

Decide  and  Recommend  /  P 

[  0480 

DYNAMIC  STRENGTH 

1 3-Recommend  Action 

f  o.ob 

0.80 

0.00 

Decide  and  Recommend  /  C 

r  0490 

TRUNK  STRENGTH 

1 1-Decide  Action 

! bob" 

0.80 

0.00 
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B 

C 

D 

_ i _ I 

1 

H 

1 

1 

M 

797 

Decide  and  Recommend  /  p  0490 

TRUNK  STRENGTH 

12-Decide 

0.00  j 

0.80 

0.00 

798 

Decide  and  Recommend  /  d  0490 

TRUNK  STRENGTH 

13-Recommend  Action 

0.00 

0.80 

0.00 

Decide  and  Recommend  /  C|  0500 

STAMINA 

1 1 -Decide  Action 

0.00 

0.80 

0.00 

800 

Decide  and  Recommend  /  ^  0500 

STAMINA 

12-Decide 

0.00 

0.80 

0.00 

801 

Decide  and  Recommend  /  p  0500 

STAMINA 

13-Recommend  Action 

0.00 

0.80 

0.00 

802 

Evaluate  and  Estimate  Impg  0010 

ORAL  COMPREHENSION 

14-Estimate  Impact 

0.00 

0.80 

0.00 

Evaluate  and  Estimate  Imp^  0010 

ORAL  COMPREHENSION 

15-Data  Gathering/  Analog 

,.2,07.. 

0.80 

1.66 

804 

Evaluate  and  Estimate  Impd  0010 

ORAL  COMPREHENSION 

16-Find  Options 

2.07 

0.80 

1.66 

805 

Evaluate  and  Estimate  Imp^  0010 
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1  2.15 

0.80 

1.72 

1107 

identify/understand  Situatio 

!  0010  1 

ORAL  COMPREHENSION 

24-Listen/Monitor  -  Analog 

I  2.15 

0.80 

1.72 

1108 

1  0610  ] 

ORAL  COMPREHENSION 

SO-Read/ Digital 

1 6l66' 

0.80 

0.00 

1109 

Identiiv/Understand  Situatio 

r  ooio'l 

ORAL  COMPREHENSION 

i  31-Scan  Digital 

1  6.66 

0.80 

0.00 

1110 

Identify/Understand  Situatio 

fool  o' 

ORAL  COMPREHENSION 

1 32-Monitor  Digital 

1  '0.66 ' 

0.80 

0.00 

1111 

Identify/Understand  Situatio 

1  0020 

WRITTEN  COMPREHENSKi  19-Read/Analog 

!  1.64 

1 

0.80 

1.31 

1112 

Identlfy/Understand  Situatio 

r  6o20  “ 

WRITTEN  COMPREHENSI^20-Scan 

1  1.64 

1— 

0.80 

1.31 

1113 

Identify/Understand  Situatio 

1  0020 

iWRlfrEN  COMPREHENSI<21-Update/  Analog 

1  1.64 

0.80 

1.31 

1114 

Identify/Understand  Situatio 

(  6626 

IWRITTEN  COMPREHENSI<i22-Check  Status 

i  1.64 

0.80 

1.31 
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II  B  1  C 

D  1  E 

F 

1 

H 

1 

1 

jiSSr 

H 

1 

Iffip 

HIM 

identify/understand  Situatioj  0020 

WRITTEN  COMPREHENSld 

23-Problem  Definition 

1.64 

0.80 

1.31 

1116 

Identify/Understand  SItuatid  0020 

WRITTEN  COMPREHENSI^ 

24-Llsten/Monitor  -  Analog 

0.00 

— 

0.80 

0.00 

1117 

Identify/Understand  Situatiq  0020 

WRITTEN  COMPREHENSld 

30-Read/  Digital 

1.64  ! 

0.80 

1.31 

Ktna 

Identify/Understand  Situatioj  0020 

WRITTEN  COMPREHENSI< 

31 -Scan  Digital 

1.64 

0.80 

1.31 

1  1119 

Identify/Understand  Situatid  0020 

WRITTEN  COMPREHENSIC 

32-Monitor  Digital 

1.64 

0.80 

1.31 

ITOil 

Identify/Understand  Situatid  0030 

ORAL  EXPRESSION 

19-Read/Analog 

0.00 

0.80 

0.00 

Identify/Understand  Situatid  0030 

ORAL  EXPRESSION 

20-Scan 

0.00 

0.80 

0.00 

Identify/Understand  Situatid  0030 

OF^L  EXPRESSION 

21 -Update/ Analog 

0.00 

0.80 

0.00 

1123 

Identify/Understand  Situatid  0030 

ORAL  EXPRESSION 

22-Check  Status 

0.00 

0.80 

0.00 

identify/understand  Situatio 

0030 

ORAL  EXPRESSION 

23-Problem  Definition 

2.49 

0.80 

1.99 

■iWil 

Identify/Understand  Situatio 

0030 

ORAL  EXPRESSION 

24-Listen/Monitor  -  Analog 

0.00 

0.80 

0.00 

Identify/Understand  Situatio 

0030 

ORAL  EXPRESSION 

30-Read/  Digital 

0.00 

0.80 

0.00 

Identify/Understand  Situatio 

6030 

ORAL  EXPRESSION 

31 -Scan  Digital 

0-00, 

0.80 

0.00 

■iw:i 

Identify/Understand  Situatio!  0030 

ORAL  EXPRESSION 

32-Monitor  Digital 

6.66 

0.80 

0.00 

■iWil 

Identify/Understand  Situatio!  0040 

WRrff EN  EXPRESSION 

19-Read/Analog 

0.00 

0.80 

0.00 

■iicHl 

Identify/Understand  Situatio!  0040 

_____ 

20-Scan 

0.00 

0.80 

0.00 

Kticn 

Identify/Understand  Situatioj  0040 

WRITTEN  EXPRESSION 

21 -Update/  Analog 

0.00 

0.80 

0.00 

BESI 

Identify/Understand  Situatic^  0040 

WRITTEN  EXPRESSION 

22-Check  Status 

0.00 

0.80 

0.00 

1133 

Identify/Understand  Situatio!  0040 

WRITTEN  EXPRESSION 

23-Problem  Definition 

6.66 

0.80 

0.00 

1134 

Identify/Understand  Situati<^  0040 

WRITTEN  EXPRESSION 

24-Listen/Monitor  -  Analog 

0.00 

L- 

0.80 

0.00 

iHcg 

Identify/Understand  Situatioj  0040 

WRITTEN  EXPRESSION 

30-Read/  Digital 

1  6.66 

0.80 

0.00 

Identify/Understand  Situatid  0040 

WRITTEN  EXPRESSION 

31 -Scan  Digital 

0.00 

0.80 

0.00 

1137 

identify/understand  Situatioj  0040 

WRITTEN  EXPRESSION 

32-Monitor  Digital 

0.00 

L 

0.80 

0.00 

nicia 

Identify/Understand  Situatio 

0050 

MEMORIZATION 

19-Read/Analog 

6.66 

0.80 

0.00 

nggi 

identify/understand  Situatio 

0050' 

MEMORIZATION 

20-Scan 

234 

0.80 

1.87 

Identify/Understand  Situatio 

0050 

[MEMORIZATION 

21 -Update/ Analog 

2.34 

0.80 

1.87 

ng-ii 

Identify/Understand  Situatio 

0050 

[memorization .  " 

22-Check  Status 

2.34 

0.80 

1.87 

identify/understand  Situatio 

0050 

MEMORiMTION 

23-Problem  Definition 

2.34 

0.80 

1.87 

■iticl 

Identify/Understand  Situatio 

6650 

24-Listen/Monitor  -  Analog 

2.34 

0.80 

1.87 

1  1144 

Identify/Understand  Situatio!  0050 

MEMORIi^fiON 

30-Read/  Digital 

2.34 

0.80 

1.87 

BEEI 

Identify/Understand  Situatioj  0050 

MEMORIZAT 

31 -Scan  Digital 

i34 

0.80 

1-87 

1  1146 

Identify/Understand  Situatioj  0050 

memori^Tion 

3^-Monitor  _ 

I  i34 

0.80 

1.87 

HEM 

Identify/Understand  Situatio*  0060 

PROBLEM  SENSlfivifY 

il9-i^adiAnaiog . . 

aoo" 

0.80 

0.00 

nggi 

tdentify/Understand  SituatM  0060 

PROBLEM  SEN^^^^^ 

2p-Scan^^_ 

T16 

0.80 

1.68 

WEEl 

Identify/Understand  Situatioj  0060 

21 -Update/  Analog^  ^ 

add 

L 

0.80 

0.00 

Identify/Understand  Situatioj  0060 

j^OBLEMsi^smvifY 

22-Check  Status 

Z16 

1 

0.80 

1.68 

1  1151 

Identify/Understand  Situatioj  0060 

1  PROBLEM  3ENs7f  ,vyfY 

23-P^  Definition 

2.10 

L 

0.80 

1.68 

niig 

identify/understand  Situatio)  0060 

24;L|sten/Monito^^^  AH® 

i  0-00 

0.80 

0.00 

nEgi 

Identify/Understand  Situatio)  0060 

jF^^OBLlM  SENSITIvi^ 

36-Read/  Digital 

I'oW 

0.80 

0.00 

1154 

Identify/Understand  Situatioj  0060 

PROBLEM  SENSITIVITY 

31 -Scan  Digital 

[6.66* 

0.80 

0.00 

1155 

identify/understand  Situatic^  0060 

32-Monitor  Digital 

UK 

. 

0.80 

1.68 

Identify/Understand  Situatic^  0070 

ORIGINALITY 

i  0.00 

0.80 

0.00 

1157 

Identify/Understand  Situatid  0070 

lORIGINALITY 

26-Scan 

I  0^06 

! 

L_ 

0.80 

0.00 

1158 

Identify/Understand  Situatioj  0070 

Ioriginality 

21 -Update/  Analog 

1  0.00* 

0.80 

0.00 

niigi 

Identify/Understand  Situatioj  0070 

jORIGINALITY 

22-Check  Status 

[  0.00" 

0.80 

0.00 

nE5i 

Identify/Understand  Situatid  0070 

IORIGINALITY 

23-Problem  Definition 

i  0.06' 

0.80 

0.00 

I  1161 

Identify/Understand  Situatid  0070 

lORIGINALITY 

24-Listen/Monitor  -  Analog 

[d66" 

t" 

0.80 

0.00 

ms 

Identify/Understand  Situatid  0070 

Ioriginality 

36-Read/  Digital 

1 d66 

1 

,ii _ 

0.80 

0.00 

WEgi 

Identify/Understand  Situatid  0070 

lORIGINALlfY 

31 -Scan  Digital 

i  0.00 

L_ 

0.80 

0.00 

■iiai 

Identify/Understand  Situatid  0070 

iOWGINALITY 

i  32-Monitor  Digital 

1  0.06 

0.80 

0.00 

KiEg 

Identify/Understand  Situatioi  0080 

1  FLUENCY  OF  IDEAS 

19-Read/Analog 

i  0.06 

0.80 

0.00 

IKBa 

Identify/Understand  Situatid  0080 

!  FLUENCY  OF  IDEAS 

_20-Scan__ . . . 

1  0.00 

0.80 

0.00 

Identify/Understand  Situatio  0080 

i  FLUENCY  OF  IDEAS 

21 -Update/ Analog 

1  0.60 

~ 

0.80 

0.00 
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B  1  C 

D 

E 

F 

i 

H 

1 

1 

1 

DeU 

iSiaSTBr, 
■iAdi . 

nRiti 

Identify/Understand  Situatiq  0080 

FLUENCY  OF  IDEAS 

22-Check  Status 

0.00 

0.80 

0.00 

nEEi 

Identlfy/Understand  Situatid  0080 

FLUENCY  OF  IDEAS 

23-Problem  Definition 

0.00 

0.80 

0.00 

HiliKfil 

Identify/Understand  Situatid  0080 

FLUENCY  OF  IDEAS 

24-Listen/Monitor  -  Analog 

0.00 

0.80 

0.00 

IIBI 

Identify/Understand  Situatid  0080 

FLUENCY  OF  IDEAS 

30-Read/  Digital 

0.00 

0,80 

0.00 

1172 

Identify/Understand  Situatid  0080 

FLUENCY  OF  IDEAS 

31 -Scan  Digital 

0.00 

0.80 

0.00 

1173 

Identify/Understand  Situatid  0080 

FLUENCY  OF  IDEAS 

32-Monitor  Digital 

6.66 

0.80 

0.00 

1174 

Identify/Understand  Situatid  0090 

FLEXIBILITY  OF  CLOSURE 

19-Read/Analog 

0.00 

0.80 

0.00 

Identify/Understand  Situatid  0090 

FLlxiiiLlfY  OF  CLOSURE 

20-Scan 

,1-4? 

0.80 

1.19 

1176 

Identify/Understand  Situatid  0090 

FLEXIBILITY  OF  CLOSURE 

21 -Update/ Analog 

0.66 

0,80 

0.00 

1177 

Identify/Understand  Situatid  0090 

FLEXIBILITY  OF  CLOSURE 

22-Check  Status 

1.49 

0.80 

1.19 

Identify/Understand  Situatio|  0090 

23-Problem  Definition 

0.00 

0.80 

0.00 

Identify/Understand  Situatid  0090 

FLEXiBILifY  OF  bCoSURE 

24-Llsten/Monitor  -  Analog 

0.00 

0.80 

0.00 

identify/understand  Situatid  0090 

FLEXIBILITY  OF  CLOSURE 

30-Read/  Digital 

0.00 

0.80 

0.00 

1  1181 

Identify/Understand  Situatid 

FLEXIBILITY  5p 

31-Scan  Digital 

0.00 

0.80 

0.00 

Identify/Understand  Situatid  0090 

FLEXiBILifY  OF  CLOSURE 

32-Monitor  Digital 

1.49 

0.80 

1.19 

1183 

Identify/Understand  Situatid  0100 

[SELECTIVE  ATTENTION 

19-Read/Analog 

1.50 

0.80 

1.20 

■iigi 

Identify/Understand  Situatio!  0100 

SELECTIVE  ATTENTION 

20-Scan 

1.50 

0.80 

1.20 

Identify/Understand  Situatid  0100 

SELECTIVE  ATTENTION 

21 -Update/ Analog 

1,56 

0.80 

1.20 

niig 

Identify/Understand  Situation  0100 

SELECTIVE  ATTENTION 

22-Check  Status 

1.50 

0.80 

1.20 

iBig 

Identify/Understand  Situatid  0100 

SELECT  IVE  ATT  ENT  ION 

23-Problem  Definition 

1.50 

0.80 

1.20 

niig 

identify/understand  Situatioj  0100 

SELECTIVE  ATTENTION 

24-Listen/Monltor  -  Analog 

1.50 

0.80 

1.20 

ni;!;! 

Identify/Understand  Situatid  0100 

SELECTIVE  AffENfToN 

30-Read/  Digital 

1.50 

0.80 

1.20 

BESI 

identify/understand  Situatid  0100 

^SELECTIVE  ATTENTION 

31 -Scan  Digital 

1.50 

0.80 

1.20 

1  1191 

Identify/Understand  Situatid  0100 

jSELECTIVE  ATTENTION 

32-Monitor  Digital 

1.50 

0.80 

1.20 

Identify/Understand  Situatioj  0110 

ISPATIAL  ORIENTATION 

.19:?J?99!^Anja!og 

0.00 

0.80 

0.00 

■iBKl 

identify/understand  Situatid  ^ 

ISPAfiALOFdE^ 

26-Scan 

qW 

0.80 

0.00 

1  1194 

identify/Understand  Situatid  01 10 

SPATIAL  ORIENW ION 

Anajog  1 

6.66 

0.80 

0.00 

liEa 

identiJ^/Un^  0110 

SPATIAL  ORIENT ATION 

22-Check  Status 

0.66 

0.80 

0.00 

ii03 

spatIal  orientation 

23-Problem  Definition 

1  6.66 

0.80 

0.00 

1  1197 

SPAfiALORlEI^^^^ 

24-Listen/Monitor  -  Analog 

i  6.00 

L 

0.80 

0.00 

IiEbI 

30-Read/  Digital 

1  0.00 

r 

s 

0.80 

0.00 

identi1^/Un  0110 

fSF^IAL  ORlirlw^ 

31 -Scan  Digital 

i  0.00 

0.80 

0.00 

nginii 

Identify/Iinderstand  Situatio!  0110 

SPATIALORE^^^ 

32^|^ifo^^  . . . . . 

s  6.66 

0.80 

0.00 

IWiH 

identify/Understand  Situatid  0120 

^SUALlEvflON . 

19-Read/Analog 

1  0.00 

, — 

0.80 

0.00 

identi^/U^^^^  Situati4^^^  0\20 

visUALi^iON 

26-Scan 

1  2.18 

0.80 

1.74 

Icientiiy/Understand  Situatid  01 20 

ViSUALIZA^iON 

21-Update/ Analog 

i  0.00 

0.80 

0.00 

Ikl»g| 

identify/Understand  Situatio!  0120 

1  visUALli^lON  “ 

22-Check  Status . 

1  2.18 

0.80 

1.74 

ipgg 

Identii^^nde^  0120 

iVISUALIZATION  I 

23-Problem  Definition 

[  Z18  ^ 

i 

0.80 

1.74 

Identify/Understand  Situatid  0120 

jvTsUALIZAfiON 

24-Listen/Monltor  -  Analog 

1  666 

0.80 

0.00 

IHiTi 

identify/Understand  Situatioj  0120 

1  VISUALIZATION .  “i 

30-Read/  Digital 

1  666' 

i 

0.80 

0.00 

TOtgj 

identify/Understand  Situatid  0120 

viSUALI^ON . .  . . j 

31-Scan  Digital 

broo 

~ 

0.80 

0.00 

IBigl 

Identify/Understand  Situatid  0120 

viSUALIZAflON 

[32-Monitor  Digital 

2.18 

0.80 

1.74 

mfii 

identify/Understand  Situatioj  0130 

INDUCTIVE  REASONING 

19-Read/Analog 

666 

0.80 

0.00 

[1211 

Identify/Understand  Situatid  0130 

INDUCTIVE  REASONING 

20-Scan 

2.59 

P_ 

0.80 

2.07 

Identify/Understand  Situatioj  0130 

ilNDUCflVE  REASONING 

21 -Update/ Analog 

1666 

0.80 

0.00 

ISEI 

Identify/Understand  Situatioj  0130 

llNDUCflVE  REASONING 

22-Check  Status 

1  2.59 

L 

0.80 

2.07 

1214 

Identify/Understand  Situatid  0130 

liNDUCTIVE  REASONING 

23-Problem  Definition 

t' 

0.80 

2.07 

1215 

Identify/Understand  Situatid  6130 

jlNDUCTIVE  REASONING 

24-Listen/Monitor  -  Analog 

666 

L 

0.80 

0.00 

mniit 

Identify/Understand  Situatid  0130 

1  INDUCTIVE  REASONING 

36-Read/  Digital 

0,00 

ij 

0.80 

0.00 

IKU 

jdentH^n^  0130 

jlNDUCTIVE  REASONING 

131 -Scan  Digital 

1  0.60 

0.80 

0.00 

BRIEi 

Identify/Understand  Situatid  0130 

I  INDUCTIVE  REASONING 

i  32-Monitor  Digital 

i  2.59 

0.80 

2.07 

■Mbi 

identify/Understand  Situatid  6l40 

[CATEGORY  FLEXIBILITY 

1 19-Read/Analog 

i  0.00 

0.80 

0.00 

I^«l 

Identify/Understand  Situatid  QUO 

ICATEGOiff  FLEXIBILITY 

1 20-Scan 

faoo^ 

0.80 

0.00 

116 
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B 

-  1  - 

E 

■Til 

H 

1 

1 

HI 

1^(xer 

iiiji 

Identify/Understand  Situatio 

0140 

CATEGORY  FLEXIBILITY 
"category  FLEXimUTY 

_ 

22-Ched<  Status 

0.00 

0.00 

0.80 

0.00 

I  1222 

Identify/Understand  SItuatid  0140 

0.80 

0.00 

Identify/Understand  Situatloj  0140 

CATEGORY  FLEXIBILITY 

23-Problem  Definition 

0.00 

0.80 

0.00 

Identify/Understand  Situatid  0140 

CATEGORY  FLEXIBILITY 

24-Listen/Monitor  -  Analog 

0.00 

0.80 

0.00 

Identify/Understand  Situatid  0140 

CATEGORY  FLEXIBILITY 

30-Read/  Digital 

0.06 

0.80 

0.00 

identify/understand  Situatid  0140 

CATEGORY  FLEXIBILITY 

31 -Scan  Digital 

0.00 

0.80 

0.00 

Identify/Understand  Situatid  0140 

CATEGORY  FLEXIBILITY 

32-Monitor  Digital 

2.58 

0.80 

2.06 

KWilii 

Identify/Understand  Situatid  0150 

DEDUCTIVE  REASONING 

19-Read/Analog 

0.00 

0.80 

0.00 

Identify/Understand  Situatio]  0150 

deductive  reasoning 

20-Scan 

2.71 

0.80 

2.17 

Identily/Understand  Situatid  0150 

DEDUCTIVE  REASONING 

21 -Update/ Analog 

0.60 

0.80 

0.00 

Identify/Understand  Situatid  0150 

deductive  reasoning 

22-Check  Status 

Z71 

0.80 

2.17 

BBcg 

Identify/Understand  Situatio!  01 50 

DEDUCTIVE  REASONING 

23-Problem  Definition 

0.00 

0.80 

0.00 

I  1233 

Identify/Understand  Situatio 

0150 

deductive  reasoning 

24-Listen/Monitor  -  Analog 

0.00 

0.80 

0.00 

BBcgl 

Identify/Understand  Situatio 

0150 

DEDUCTIVE  REASONING 

30-Read/  Digital 

0.00 

0.80 

0.00 

iBcg 

Identify/Understand  Situatio 

0150 

deduct IVE  REASONING 

31 -Scan  Digital 

0.00 

0.80 

0.00 

Bggg 

Identify/Understand  Situatio 

0150 

DEDUCTIVE  REASONING 

32-Monitor  Digital 

2.71 

0.80 

2.17 

KBcg 

Identify/Understand  Situatio]  0160 

INFORMATION  ORDERING 

19-Read/Analog 

0.00 

0.80 

0.00 

IBcgj 

Identify/Understand  Situatio]  0160 

INFORMATION  ORdIrINQ 

20-Scan 

0.00 

0.80 

0.00 

Identify  Understand  Situatid  0160 

INFORMATION  ORDERING21-Update/  Analog 

2.89 

0.80 

2.31 

■Mtll 

Identify/Understand  Situatid  0160 

IIMFORMAT ION  ORDERING  22-Check  Status 

239 

0.80 

2.31 

Identify/Understand  Situatio]  0160 

INFORMAT fON  ’6RDERiN023-Probleni  Definition 

2.89 

0.80 

2.31 

IHkl 

Identify/Understand  Situatid  0160 

rNFORMATION  ORDERINQ‘24-Lis"ten/Monltor  -  Analog 

0.00 

0.80 

0.00 

Bkgki 

Identify/Understand  Situatid  0160 

INFORMATION  ORDERINC^  30-Read/  Digital 

0.00 

0.80 

0.00 

Identify/Understand  Situatid  0160 

INFORMATION  ORDERING  31 -Scan  Digital 

0.00 

0.80 

0.00 

KBgg 

Identify/Understand  Situatid  0160 

Information  ordering  32-Monitor  Digital 

2.89 

0.80 

2.31 

BkgM 

Identify/Understand  Situatid  0170 

MATHEMATICAL  REASONI 

19-Read/Analog 

0.06 

0.80 

0.00 

L!2^ 

Identify/Understand  Situatid  0170 

MATHEMATICAL  REASONI 

20-Scan 

il4 

E_ 

0.80 

1.71 

KBekI 

Identify/Understand  Situatid  0170 

MATHEMATICAL  REASONI 

21 -Update/ Analog 

0.00 

0.80 

0.00 

Bkgbl 

Identify/Understand  Situatio]  0170 

MATHEMATICAL  REASONI 

22-Check  Status 

2.14 

0.80 

1.71 

KHiBI 

Identify/Understand  Situatid  0170 

MATHEMATICAL  REASONI 

23-Problem  Definition 

6.00 

h 

0.80 

0.00 

TOin 

Identify/Understand  Situatid  0170 

MATHEMATICAL  REASONI 

24-Listen/Monltor  -  Analog 

6^66 

0.80 

0.00 

Identify/Understand  Situatid  0170 

MATHEMATICAL  REASONI 

30-Read/  Digital 

aoo" 

0.80 

0.00 

BH3cl 

Identify/Understand  Situatid  0170 

MMHEMAT IC^  REASONI 

31-Scan  Digital 

6.00 

( 

L 

0.80 

0.00 

»Wg-l 

Identify/Understand  Situatid  0170 

reason! 

32-Monitor  Digital 

T1F1 

0.80 

1.71 

BHig 

Identify^^ 

Identify/Understand  Situatid  0180 

NUMBER  FACILITY 

19-Read/Analog 

6.06 

0.80 

0.00 

BHdJ 

NUMBER  FACILITY 

20-Scan 

i  2.50 

0.80 

2.00 

BH.-VI 

Identify/Understand  Situatio]  0180 

NUMBER  FACILITY 

21 -Update/  Analog 

i  o.dd 

0.80 

0.00 

BFiig 

Identify/Understand  Situatid  0180 

NUMBER:  FACILITY 

22-Check  Status 

(om' 

0.80 

0.00 

BBiBI 

Identify/Understand  Situatio!  0180 

NUMBER  FACILITY 

23-Problem  Definition 

0.00"' 

0.80 

0.00 

IMAl 

Identify/Understand  Situatid  0180 

NUMBER  FACILITY 

24-Listen/Monitor  -  Analog 

i  0.00 

0.80 

0.00 

BBSn 

NUMBER  FACILITY 

30-Read/  Digital 

i  6.60 

0.80 

0.00 

BRig 

Identify/Understand  Situatio]  0180 

NUMBER  FACILITY 

31 -Scan  Digital 

I  0.00 

L 

0.80 

0.00 

B^sg 

Identify/Understand  Situatid  0180 

NUMBER  FACILITY 

32-Monitor  Digital 

1  2.50  ' 

0.80 

2.00 

EVl^i 

Identify/Understand  Situatid  0190 

TIME  SHARING 

19-Read/Analog 

i  2.00 

j 

0.80 

1.60 

BFgg 

Identify/Understand  Situatid  0190 

TIME  SHARING 

20-Scan 

i'2.d6 

0.80 

1.60 

BRBg 

Identify/Understand  Situatid  0190 

TIME  SHARING 

21 -Update/ Analog 

1  2M \ 

0.80 

1.60 

BEsa 

Identify/Understand  Situatid  0190  | 

TIME  SHARING 

22-Check  Status 

2M 

i 

0.80 

1.60 

IMAl 

01 90 

TIME  SHARING 

23-Problem  Definition 

2.00 

n 

L 

0.80 

1.60 

BRiBl 

Identify/Understand  Situation  0190 

ITIME  SHARING 

24-Listen/Monitor  -  Analog 

2^0 

0.80 

1.60 

PiMtl 

Identify/Understand  Situatio'  0190 

TIME  SHARING 

30-Read/  Digital 

2.00  1 

0.80 

1.60 

Bkm 

IdentiiyUnderstand  Situatid  0190 

TIME  SHARING 

31-Scan  Digital 

2.00  i 

0.80 

1.60 

B^a 

Identify/Understand  Situatid  0190 

TIME  SHARING 

32-Monitor  Digital 

2.00  i 

0.80 

1.60 

BR?g| 

Identify/Understand  Situatid  0200 

SPEED  OF  CLOSURE 

19-Read/Analog 

0.60 1 

0.80 

0.00 
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B 

C 

D  E 

zn 

q 

PEP 

1 

B 

Ti 

Identify/Understand  Situatiq  0200 

SPEED  OF  CLOSURE 

20-Scan 

2.22  j 

0.80 

1.78 

Identify/Understand  Situatioi  0200 

SPEED  OF  CLOSURE 

21-Update/ Analog 

0.66  ] 

0.80 

0.00 

IRH 

Identify/Understand  Situatio!  0200 

SPEED  OF  CLOSURE 

22-Check  Status 

2.22 

0.80 

1.78 

wtsn 

Identify/Understand  Situatioj  0200 

SPEED  OF  CLOSURE 

23-Problem  Definition 

2.22  i 

0.80 

178 

Identify/Understand  Sltuatic^  0200 

SPEED  OF  CLOSURE 

24-Listen/Monitor  -  Analog 

2.22 1 

0.80 

178 

identify/understand  Situatioj  0200 

SPEED  OF  CLOSURE 

30-Read/  Digital 

0.00 

0.80 

0.00 

g^Iil 

Identify/Understand  Situatioj  0200 

SPEED  OF  CLOSURE 

31-Scan  Digital 

0.00 

0.80 

0.00 

Identify/Understand  Situatioi  0200 

SPEED  OF  CLOSURE 

32-Monitor  Digital 

2.22 

0.80 

1.78 

Identify/Understand  Situatioi  0210 

PERCEPTUAL  SPEED  AND 

19-Read/Analog 

0.00 

0.80 

0.00 

Identify/Understand  Situatioj  0210 

PERCEPTUAL  SPEEDANq 

20-Scan 

2.36 

0.80 

1.89 

■W;^l 

identify/understand  Situatioj  0210 

SPEED  Km 

21-Update/  Analog 

2.36 

0.80 

189 

Identify/Understand  Situatioj  0210 

PERCEPTUAL  SPEED  AND 

22-Check  Status 

2.36 

0.80 

189 

Identify/Understand  Situatioj  0210 

PERCEPTUAL  SPEED  ANC^ 

23-Problem  Definition 

0.60 

0.80 

0.00 

ren 

Identify/Understand  Situatic^  0210 

PERCEPTUAL  SPEED  AND 

24-Listen/Monltor  -  Analog 

2.36 

0.80 

1.89 

Identify/Understand  Situatioj  0210 

PERCEPTUAL  SPEED  AND 

30-Read/  Digital 

6.66 

0.80 

0.00 

re;g| 

Identify/Understand  Situatio|  0210 

PERCEPTUAL  SPEED  AND 

31 -Scan  Digital 

0.00 

0.80 

0.00 

g^ 

Identify/Understand  Situatid  0210 

32-Monltor  Digital 

2.36 

0.80 

1.89 

gggn 

Identify/Understand  Situatioi  0220 

REACTION  TIME 

19-Read/Analog 

0.00 

0.80 

0.00 

g^g 

identify/understand  Situatio|  0220 

REACTION  TIME 

20-Scan 

0.00 

0.80 

0.00 

ggigl 

Identify/Understand  Situatioj  0220 

REACTION  TIME 

21-Update/  Analog 

0.00 

0.80 

0.00 

g^gl 

Identify/Understand  Situatioi  0220 

reaction  TIME 

22-Check  Status 

0.00 

0.80 

0.00 

gkbbi 

Identify/Understand  Situatioi  0220 

REACTION  TIME 

23-Problem  Definition 

0.00 

0.80 

0.00 

gKBa 

Identify/Understand  Situatid  0220 

rIaCTION  flME 

24-Listen/Monltor  -  Analog 

0.00 

0.80 

0.00 

gFISi 

Identify/Understand  Situatid  0220 

1  REACTION  TIME 

30-Read/  Digital 

0.00 

0.80 

0.00 

gFga 

Identify/Understand  Situatioj  0220 

REACTION  TIME 

31-Scan  Digital 

0.00 

0.80 

0.00 

gFggl 

Identify/Understand  Situatio!  0220 

REACTION  TIME 

32-Monitor  Digital 

0.00 

0.80 

0.00 

gE!ita 

Identify/Understand  Situatic^  0230 

CHOICE  REACTION  TIME 

19-Read/Analog 

0.00 

0.80 

0.00 

g|t!gl 

Identify/Understand  Situatio!  0230 

CHOICE 

20-Scan 

6.66 

0.80 

0.00 

glc!ig 

Identify/Understand  Situatid  0230 

CHOICE  REACTION  TIME 

21 -Update/  Analog 

0.00 

0.80 

0.00 

g|c!«itl 

Identif^Understand  Situatid  0230 

SCHOICE  REACTION 

22-Check  Status 

1 . a66"i 

r 

0.80 

0.00 

gPcWi 

CHOICE  REACTION  TIME 

1  6.06 

j 

j 

0.80 

0.00 

gic?tg 

jidentify^ 

CHOICE  REACTION  TIME 

24-Listen/Monitor  -  Analog 

1  6.60 

0.80 

0.00 

gRia 

identify/U^  ?!f^3tio|  0230 

CHOICE  pg^fjoN  time 

36-Read/ Digitaj^^^^ 

1  6.00 

0.80 

0.00 

gg!»g 

fdentify/Unde^^^ 

3VScan_pjgita] 

0.80 

0.00 

gE!*!! 

Identify/Und^  02Z0 

CHOICE  REA^^^^^^^ 

32-Monitor^  _ 

j  o.‘6d 

r~ 

0.80 

0.00 

gMfel 

iderrtify^ 

19-Reacl/Analog 

U-K 

p 

0.80 

0.98 

■Kittl 

identify/Understand  Situatid  0240 

NEAR  VISION 

26;Scan  .  . . . . . . . . 

1 1.22 

0.80 

0.98 

gHtI 

Identify/Und^^^ 

NEAR  VISION 

21  -Update/ 

0.80 

0.98 

ggra 

Identify/Understand  Situatio|  0246 

. . . 

22-Check  Status 

I’o.dd'" 

0.80 

0.00 

ggna 

Identify/UnderstancI  Situatid  6240 

NEARVisiON 

23-Problem  Definition 

j  a  dd 

0.80 

0.00 

I  1314 

Identify/Understand  Situatioj  6240 

NEAR  VISION 

24-Listen/Monitor  -  Analog 

j  0^00" 

0.80 

0.00 

ISSI 

Identify/Understand  Situatio!  6246 

NEARVISiON 

30-Read/  Digital 

I  122 

0.80 

0.98 

IgMM 

Identify/Understand  Situatioj  0240 

NEARVISION 

31 -Scan  Digital 

1.22 

0.80 

0.98 

I  1317 

Identify/Understand  Situatid  6246 

NEARVfsION 

32-Monitor  Digital 

122 

0.80 

0.98 

ggna 

Identify/Understand  Situatio;  0250 

FAR  VISION 

lO-Read/Analog 

0.00 

0.80 

0.00 

gHEl 

Identify/Understand  Situatid  0256 

FARVlsiON 

20-Scan 

6.66 

0.80 

0.00 

gB^ 

identify/Understand  Situatid  0250 

1  FAR  VISION 

21 -Update/ Analog 

r  aoo 

0.80 

0.00 

gBEtl 

identify/Understand  Situatid  0250 

FAR  VISION 

22-Check  Status 

j  1.08 

p 

0.80 

0.86 

■ICW>J 

Identify/Understand  Situatid  0250 

FAR  VISION 

23-Problem  Definition 

faoo 

i 

0.80 

0.00 

gEBtl 

identify/Understand  Situatid  6256 

FAR  VISION 

24-Listen/Monitor  -  Analog 

raoo 

p 

0.80 

0.00 

gEB?l 

identify/Understand  Situatioj  0256 

FAR  VISION 

1 30-Read/ Digital  i  0.00 

0.80 

0.00 

gEBa 

Identify/Understand  Situatia  0250 

FAR  VISION 

31 -Scan  Digital  j  0.00  j 

0.80 

0.00 

gEga 

identify/Understand  Situatid  0256 

FAR  VISION 

1 32-Monitor  Digital  j  0.00  1 

0.80 

0.00 
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B  1  C 

D 

_ i _ 1 

F 

G 

H 

“n 

1 

;; 

s 

^8 

M 

Identify/Understand  Situatioi  0260 

NIGHT  VISION 

19-Read/Analog 

0.00 

0.80 

0.00 

Bcgil 

identify/understand  SituatiQ^  0260 

NIGHT  VISION 

20-Scan 

6,66 

0.80 

0.00 

g|c^ 

Identify/Understand  Situatic^  0260 

NIGHT  VISION 

21 -Update/ Analog 

0.00 

0.80 

0.00 

IBgil 

Identify/Understand  Situatid  0260 

NIGHT  VISION 

22-Check  Status 

0.00 

0.80 

0.00 

Bcgn 

Identify/Understand  Situatioj  0260 

NIGHT  VISION 

23-Problem  Definition 

0,00 

0.80 

0.00 

Bcicigi 

Identify/Understand  Situatid  0260 

NIGHT  VISION 

24-Listen/Monitor  -  Analog 

0.00 

0.80 

0.00 

iBcg 

Identity/Understand  Situatid  0260 

NIGHT  VISION 

30-Read/  Digital 

0.00 

0.80 

0.00 

Hcggi 

Identify/Understand  Situatid  0260 

NIGHT  VISION 

31 -Scan  Digital 

0.00 

0.80 

0,00 

Identify/Understand  Situatid  0260 

NIGHT  VISION 

32-Monitor  Digital 

0.00 

0.80 

0.00 

Identify/Understand  Situatid  0270 

VISUAL  COLOR  DISCRIMlN 

Vo-Read/Analog 

0.00 

0.80 

0.00 

1337 

Identify/Understand  Situatid  0270 

VISUAL  COLOR  DISCRiMIN 

20-Scan 

1.50 

0.80 

■ESI 

Identify/Understand  Situatid  0270 

VISUAL  COLOR  DISCRiMliN 

21 -Update/ Analog 

1.50 

0.80 

EtXSi 

Identify/Understand  siituatioj  0270 

viSLI/i  COLOR  DISCRIMIf^ 

22-Check  Status 

1.56 

0.80 

Kljil 

■wni 

Identify/Understand  Situatic^l  0270 

VISUAL  COLOR  DISCRIMIN 

23-Problem  Definition 

6.66 

0.80 

BHBfil 

gtgn 

Identify/Understand  Situatioi  0270 

VISUAL  COLOR  DISCRIMIN 

24-Listen/Monitor  -  Analog 

0.00 

0.80 

■ESI 

Identify/Understand  Situatio!  0270 

VISUAL  COLOR  DISCRImI^ 

30-Read/  Digital 

1.50 

0.80 

ngRi 

Identiiy/Understand  Situatio!  0270 

VISUAL  COLOR  DISCRIMIh 

31 -Scan  Digital 

1.50 

0.80 

1.20 

Identify/Understand  Situatid  0270 

VISUAL  COLOR  DISCRIMIh 

32-Monltor  Digital 

1.50 

L 

0.80 

1.20 

pgg 

Identify/Understand  Situatioj  0280 

PERIPHERAL  VISION 

19-Read/Analog 

b.oo 

0.80 

0.00 

Bgni 

Identify/Understand  Situatioj  0280 

PERIPHERAL  VISION 

26-Scan 

0.00 

0.80 

lEgg 

tdentify/Understand  Situatioj  0280 

PERIPHERAL  VISION 

21 -Update/ Analog 

0.00 

0.80 

lK?;i 

Identify/Understand  Situatioj  0280 

PERIPHERAL  VISION 

22-Check  Status 

0.00 

0.80 

Identify/Understand  Situatioj  0280 

PERIPHERAL  VISION 

23-Problem  Definition 

0.00 

0.80 

Identify/Understand  Situatioj  0280 

PERIPHERAL  visiO~N 

24-Listen/Monitor  -  Analog 

0.00 

0.80 

1351 

identify/understand  Situatic^,  0280 

PERIPHERAL  VISION 

30-Read/  Digital 

0.00 

0.80 

Identiiy/Understand  Situatic^  0280 

31 -Scan  Digital 

0.00 

0.80 

0.00 

■KWCl 

Identify/Understand  Situatioi  6280 

PERIPHEI^LVIS^^^ 

32-Monitor  Digital 

0.00 

0.80 

Hcgi 

Identify/Understand  Situatioj  0296 

DEPTHPER^^  . 

19-Read/Analog 

0.00 

0.80 

■«»«!*■ 

Identify/Understand  Situatioj  0290 

[DEPTH  PERCEPTION  | 

20-Scan 

0.00 

L 

0.80 

i  DEPTH  PERCEPflOrsI  " 

21 -Update/ Analog 

666 

I 

0.80 

iRig 

6290 

DEPTH  F^RCEFTION 

22-Check  Status 

1.66 

i 

0.80 

Bcgg 

IdentiJ^nde^^ 

. 

23-Problem  Definition 

066 

0.80 

jdenUf^ni^ 

24-Listen/^^^^^ 

“6.06' 

[I 

0.80 

KMrfJ 

Identif^Understand  Situate  0290 

66Ftaad/  Digital  . . . 

”6^6' 

1 

0.80 

IB31 

jdentify/Un 

i  DEPTH  PERCEPTION 

i^i9‘t®i . 

1  o.oo' 

1 

0.80 

IMtVJ 

Ident^/Undere 

62-Monltor  Digital 

j  6.60 

L 

0.80 

IMjei 

Wentij^/Understan^  6300 

_____  . 

. . 

19-Reacl/Analog 

1  6.00 

0.80 

WBIil 

Kti!S"i 

Tdentij^/Unde  P?o6 

26-Scan . . . . 

i  o’oo' 

0.80 

Itkt^-i 

Identity/Understand  Situatioj  6666 

GLARE  SENSITIVITY 

21 -Update/ Analog 

fo.oo" 

0.80 

Pcfsa 

identil^ij^ 

GLARE  SENSITIVITY 

22-Check  Status 

16  Jo 

0.80 

■ESI 

iHsg 

Identif^VUnderstand  Situatid  0366 

GLARE  SENSITIVITY 

23-Problem  Definition 

i 6J6 

L 

0.80 

iic!gi 

Identi^Understand  Situatioj  0300 

GLARE  SENSITIVITY 

24-Usten/Monl^^^^^^ 

i  6.66 

0.80 

■ESI 

Klcggl 

Identify/Understand  Situatid  6666 

GLARE  SENSITIVITY 

66-Read/ pjgital 

1  0.06 

L 

0.80 

0.00 

PcHl 

Identify/Understand  Situatid  0300 

GLARE  SENSITIVITY 

31 -Scan  Digital 

j  6.00 

1 

0,80 

0.00 

imi 

GLARE  SENSITIVITY 

62-Monitor  Digital 

1  0.00 

! 

0,80 

Identify/Understand  Situatiq  031 6 

[GENERAL  HEARING 

19-Read/Anaiog 

1  0.06 

0.80 

BcBci 

Identify/Understand  Situatid  031 6 

GENERAL  HEARING 

20-Scan 

1  0.00 

L. 

0.80 

■ESI 

ikVZI 

Identify/Understand  Situatio  031 6 

iGENERAL  HEARING 

21 -Update/ Analog 

!  6.00 

0.80 

Iggi 

Identi^/Underste  031 6 

[GENERAL  HEARING 

22-Check  Status 

i  6.00 

\ 

0.80 

KE^ 

identify/Understand  Situatio  0310 

GENERAL  HEARiNG . 

23-Problem  Definition 

j  a66“ 

1 

0.80 

HtkiW 

lECT 

identify/understand  Situatid  031 6 

GENERAL  HEARING 

24-Listen/Monitor  -  Analog 

ioJo 

j 

0.80 

lkW;l 

Identify/Understand  Situatioj  031 6 

GENERAL  HEARING 

30-Read/  Digital 

1  0.00 

0.80 

BJtM 

lESS 

Identify/Understand  Situatioi  031 6 

GENERAL  hiring 

31 -Scan  Digital 

1  6.66 

0.80 
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B  1  C 

D 

E 

F 

G 

H 

1 

1 

IP® 

Identify/Understand  Situatiol  0310 

GENERAL  HEARING 

32-Monitor  Digital 

0.00 

0.00 

ilcCTI 

Identify/Understand  Situatid  0320 

AUDITORY  ATTENTION 

19-Read/Analog 

0.00 

0.80 

0.00 

Identify/Understand  Situatid  0320 

AUDITORY  ATTENTION 

20-Scan 

0.00 

0.80 

0.00 

Identify/Understand  Situatid  0320 

AUDITORY  ATTENTION 

21 -Update/ Analog 

0.00 

0.80 

0.00 

I  1384 

identify/Understand  Situatid  0320 

AUDITORY  ATTENTION 

22-Check  Status 

0.00 

0.80 

0.00 

Identify/Understand  Situatid  0320 

AUDITORY  ATTENTION 

23-Problem  Definition 

1.96 

0.80 

1.57 

■KMil 

identify/understand  Situatid  0320 

AUDITORY  ATTENTION 

24-Listen/Monitor  -  Analog 

1.96 

0.80 

1.57 

Identify/Understand  Situatid  0320 

AUDITORY  ATTENTION 

36-Read/  Digital 

0.00 

0.80 

0.00 

Identify/Understand  Situatid  0320 

AUDITORY  ATTENTION 

31 -Scan  Digital 

6.06 

0.80 

0.00 

KHigl 

identify/understand  Situatid  0320 

AUDITORY  ATTENTION 

32-Monitor  Digital 

0.00 

0.80 

0.00 

lEgta 

Identify/Understand  Situatid  0330 

SOUND  LOCALI^TION 

19-Read/Analog 

0.00 

0.80 

0.00 

Kicgn 

Identify/Understand  Situatid  0330 

SOUND  LOCALIZATION 

20-Scan 

0,00 

0.80 

0.00 

KRgg 

Identify/Understand  Situatid  0330 

SOUND  LOCALIZATION 

21 -Update/  Analog 

0.00 

0.80 

0.00 

Bcga 

Identify/Understand  Situatid  0330 

SOUND  LOCALIZATION 

22-Check  Status 

0.00 

0.80 

0.00 

Bcgg 

Identify/Understand  Situatid  0330 

SOUND  LOCALIZATION 

23-Problem  Definition 

0.06 

0.80 

0.00 

lEggi 

Identify/Understand  Situatid  0330 

SOUND  LOCALIZATION 

24-Listen/Monitor  -  Analog 

0.00 

0.80 

0.00 

Begin 

Identify/Understand  Situatid  0330 

SOUND  LOCALIZATION 

30-Read/  Digital 

0.00 

0.80 

0.00 

Bcgg 

Identify/Understand  Situatid  0330 

SOUND  LOCALIZATION 

31 -Scan  Digital 

0.00 

0.80 

0.00 

Bcga 

Identify/Understand  Situatid  0330 

SOUND  LOCALIZATION 

32-Monitor  Digital 

6.66 

0.80 

0.00 

Identify/Understand  Situatiol  0340 

CONTROL  PRECISION 

19-Read/Analog 

0,00 

0.80 

0.00 

lEMtl 

Identiiy/Understand  Situatiol  0340 

CONTROL  PRECISION 

20-Scan 

0,00 

0.80 

0.00 

IHiH 

Identify/Understand  Situatid  0340 

CONTROL  PRECISION 

21 -Update/ Analog 

0.00 

0.80 

0,00 

■Eitm 

Identify/Understand  Situatid  0340 

CONTROL  PRECISION 

22-Check  Status 

0.00 

0.80 

0.00 

iRgcl 

Identify/Understand  Situatic^  0340 

CONTROL  PRECISION 

23-Problem  Definition 

0.00 

0.80 

0.00 

gngi 

Identify/Understand  Situatid  0340 

CONTROL  PRECISION 

24-Listen/Monitor  -  Analog 

0.00 

0.80 

0.00 

KRiin 

Identify/Understand  Situatid  0^40 

ICONT  ROL  PRECISION 

36-Read/  Digital 

0.00 

0.80 

0.00 

lEwn 

iCONTROL  PRECISION 

0.00 

0.80 

0.00 

■EWJ 

Identify/Understand  Situatid  0^40 

CONTROL  PRECISION 

32-Monitor  Digitai 

0.00 

0.80 

0.00 

KEIilil 

Irate  control 

l^?®®5!j!Analog 

0.00 

S 

0.80 

0.00 

■E!W1 

Identify/Understand  Situatia;  0350 

I  F^E  CONTROL . .  . 

20-Scan 

1  6.00 

S 

0.80 

0.00 

1410 

iF^TE  control 

21 -Update/ Analog 

1  0.00 

r 

0.80 

0.00 

1411 

Identify 

|F6\fE  CONTROL 

22-Check  Status 

1  0.00 

1 

0,80 

0.00 

man 

Identify/Understand  Situatid  0350 

IRATE  CONTROl 

23-Probiem^  ! 

1  0.00 

0.80 

0.00 

lEBcl 

identify/understand  Situati^^  0350 

. 

24-Usten/Monitor I 

fo.oo 

t 

0.80 

0.00 

1414 

Identify/Understand  Sltuatio[  0350 

[f^c^TE  CONTROL . . . 

36-Rea^  y ..  i 

1  6.06^ 

1 

: 

0.80 

0.00 

1415 

[dentify/Uiide^^^ 

if^iVTicONTROL’^'''’ 

1  0.00 

1 

0.80 

0.00 

IEII3 

. . 

32-Mqni^^^^^  j 

1  6.60  ‘ 

L 

0.80 

0.00 

1  1417 

Identify/U nderstand  Situatid  0366 

wSsf-FINGER  SPEED 

19-Read/Analog  i 

i  6.66 

0.80 

0.00 

IBEl 

Identify/Understand  Situatia  0360 

1  wrist-finger  spIed 

26-Scan 

I  6.6o~ 

r 

0.80 

0.00 

KPrei 

Identify/U nde  Situatid  0360 

(WRIsf-FiNGERSPEED 

21 -Update/ Analog 

1  0.00 

1 _ j 

0.80 

0.00 

iBsa 

Identify/Understend  Situatid  0360 

iWRIST-FINGER  SPEED 

22-Check  Status 

f^aoo' 

M 

0.80 

0.00 

lEHl 

Identify/Understand  Situatiq  0360 

IWRIST-FINGER  SPEED 

23-Problem  Definition 

ra66 

0.80 

0.00 

1  1422 

Identify/Understand  Situatid  6360 

IWRIST-FINGER  SPEED 

24-Listen/Monitor  -  Analog 

i  0.00 

0.80 

0.00 

IBgl 

Identify/Understand  Situatid  0360 

jWRIST-FINGER  SPEED 

36-Read/  Digital 

1  1.00 

0.80 

0.80 

EE^ 

Identify/Understand  Situatid  0360 

iwRIST-FINGER  SPEED 

31 -Scan  Digital 

i  1-00 

i 

0,80 

0.80 

iggg 

Identify/Understand  Situatioj  0360 

[WRISf-FINGER  SPEED 

32-Monitor  Digitai 

]  6.66 

r 

0.80 

0.00 

ig^ 

Tdentify/Understand  Situatioj  0370 

IFINGER  DEXTERITY 

19-Read/Analog 

i  6.60 

0.80 

0.00 

iggg 

Identify/Understand  Situatioj  0370 

1  FINGER  DEXTERITY 

20-Scan 

f  0.66 

0.80 

0.00 

igggi 

Identify/Understand  Situatiol  0370 

IFINGER  DEXTERITY 

21-Update/  Analog 

j  1.63 

0.80 

0.82 

Identify/Understand  Situatid  0370 

!  FINGER  DEXTERITY 

22-Check  Status 

i  6.66 

L 

0.80 

0.00 

IBcIil 

Identify/Understand  Situatid  0370 

[finger  dexterity 

23-Problem  Definition 

1  0.00 

0.80 

0.00 

lEgn 

1  Identify/Understand  Situatid  0370  | FINGER  DEXTERITY 

24-Listen/Monitor  -  Analog 

1 6H 

L 

0.80 

0.00 

BIBl 

Identify/Understand  Situatid  0370 

FINGER  DEXTERITY 

30-Read/  Digitai 

1  1.03 

I 

0.80 

0.82 
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B 

C 

_ D _ 

E  - 1 

q 

H 

1 

1 

ii^^P 

5 

lEEEl 

identify/understand  Situatio 

0370 

FINGER  DEXTERITY 

31 -Scan  Digital 

.1.03  , 

0.80 

■PI 

1434^ 

Identify/Understand  Situatio 

0370 

FINGER  DEXTERITY 

32-Monitor  Digital 

0.00 

0.80 

■iliM 

Identify/Understand  Situatio 

0380 

MANUAL  DEXTERITY 

19-Read/Analog 

0.00 

0.80 

lEHil 

Identify/Understand  Situatio 

0380 

MANUAL  DEXTERITY 

20-Scan 

0.00 

0.80 

Identify/Understand  Situatioj  0380 

MANUAL  DEXTERITY 

21 -Update/ Analog 

1.21 

0.80 

EES 

Identify/Understand  Situatid  0380 

MANUAL  DEXTERITY 

22-Check  Status 

0.00 

0.80 

0.00 

lEltgl 

Identify/Understand  Situatio!>  0380 

MANUAL  DEXTERITY 

23-Problem  Definition 

0.00 

0.80 

0.00 

iggia 

Identify/Understand  Situatio!  0380 

MANUAL  DEXTERITY 

24-Listen/Monitor  -  Analog 

0.00 

0.80 

0.00 

KM 

Identify/Understand  Situatio!  0380 

MANUAL  DEXTERITY 

30-Read/  Digital 

1.21 

0.80 

Identify/Understand  Situatid  0380 

MANUAL  DEXTERITY 

31 -Scan  Digital 

1.21 

0.80 

■*yj 

Iggkl 

Identify/Understand  Situatid  0380 

MANUAL  DEXTERITY 

32-Monitor  Digital 

0.00 

0.80 

Bifiltil 

Identiiy/Understand  Situatid  0390 

ARM-HAND  STEADINESS 

19-Read/Analog 

0.00 

0.80 

EES 

identify/understand  Situatid  0390 

ARM-HAND  STEADINESS 

20-Scan 

0.00 

0.80 

0.00 

CSS 

Identify/Understand  Situatio!  0390 

ARM-HAND  STEADINESS 

21 -Update/  Analog 

0.00 

0.80 

0.00 

1  1447 

Identify/Understand  Situatid  0390 

ARM-HAND  ST iAOlNESS 

22-Check  Status 

0.00 

0.80 

KggI 

Identify/Understand  Situatiq  0390 

ARM-HAND  STEADINESS 

23-Problem  Definition 

0.00 

0.80 

KgEl 

Identi^Understand  Situatid  0390 

ARM-HAND  STEADINESS 

24-Listen/Monitor  -  Analog 

0.00 

0.80 

■EWtl 

Identify/Understand  Situatid  0390 

ARM-HAND  STEADINESS 

30-Read/  Digital 

0.00 

0.80 

EES 

KHl 

Identify/Understand  Situatio;  0390 

ARM-HAND  STEADINESS 

31 -Scan  Digital 

0.00 

0.80 

EES 

Kcg 

identify/understand  Situatid  0390 

ARM-HAND  STEADINESS 

32-Monitor  Digital 

0.00 

0.80 

lE^ 

Identify/Understand  Situatioj  0400 

MULTI-LIMB  COORDINATI<5 

19-Read/Analog 

6.66 

0.80 

I  1454 

Identify/Understand  Situatid  0400 

MULTI-LIMB  cdORDiNAT  1^ 

20-Scan 

0.00 

0.80 

W»Wil 

Kgg 

Identify/Understand  Situatio  0400 

MULTI-LIMB  COORDINATIC 

21 -Update/ Analog 

0.00 

0.80 

EES 

KEa 

Identify/Understand  Situatio!  0400 

MULTI-LIMB  COORDINATIC 

22-Check  Status 

0.00 

0.80 

EES 

lEUVi 

Identify/Understand  Situatid  0400 

MULTI-LIMB  COORDINATIC 

23-Problem  Definition 

0.00 

0.80 

0.00 

KEia 

Identify/Understand  Situatioj  0400 

MULTI-LIMB  COORDINATIC 

24-Listen/Monitor  -  Analog 

0.00 

0.80 

EES 

K!i51 

Identify/Understand  Situatid  0400 

MULTI-LIMB  COORDINATIC 

30-Read/  Digital 

0.00 

0.80 

0.00 

Identify/Understand  Situatid  0400 

MUlfi-LIMB  cdORDIlwrc 

31 -Scan  Digital 

0.00 

0.80 

0.00 

KEHI 

Identify/Understand  Situatioj  0400 

MULTI-LIMB  COORDINATIC 

32-Monitor  Digital 

0.00 

0.80 

0.00 

Kgg 

Identify/Understand  Situatid  0410 

EXTENT  FLEXiiiUTY 

19-Read/Analog 

aoo 

0.80 

0.00 

K!ig 

Identify/Understand  Situatioj  0410 

EXTENT  FLEXIBILITY 

20-Scan 

0.00  ^ 

0.80 

0.00 

Identify/Understand  Situatioj  0410 

BCTENT  FLEXIBILITY 

21 -Update/ Analog 

0.00  1 

i 

0.80 

0.00 

KEa 

identify/understand  Situatio[  0410 

22-^heck  Status  __  J 

1l66  : 

I _ 

0.80 

0.00 

KEa 

identify/understand  Situatid  6410 

EXTENT  FLEXIBILITY 

2  3:  P  rqb  l  e  m  ^  I 

0.80 

0.00 

Kia 

Identify/Understand  Situatio 
Identify/Understand  Situatio 

j  0410 

24-Listen/Monito^^^ 

i  0.00 

0.80 

0.00 

lEtAl 

0410 

EXTENT  FLEXIBILITY 

30-Read/  Digita] . .  . . . . 

i  0.00 

0.80 

0.00 

KTsfe'l 

Identify/Understand  Situatid  0410 

iiaENTFLEXiBILiTY 

1  0.00 

L 

0.80 

0.00 

KCTl 

Identify/Understand  Situatioj  0410 

_ 

32-IVoiiit9r  Digital 

1  aoo 

! _ 

0.80 

0.00 

lEMI 

Identify/Understand  Situatid  0420 

DYNAMIC  FLEXIBILITY 

19-Read/Analog 

\K6o 

0.80 

0.00 

1472 

Identify/ltnderstand  Situatioj  0420 

DYNAMIC  pL^jgiLify 

20-Scan 

!  6.66“ 

0.80 

0.00 

1473 

Identify/Understand  Situatioj  0420 

DYNAMic  FLEXIBILITY 

21 -Update/ Analog 

j  6.00 

0.80 

KM 

Identify/Understand  Situatioj  0420 

DYNAMIC  FLEXIBILI^ 

22-Check  Status 

6.66 

0.80 

0.00 

matii 

Identify/Understand  Situatioj  0420 

IdYNAMIC  FLEXIBILITY 

23-Problem  Definition 

1 6.66 

L 

0.80 

0.00 

IEE3 

Identify/Understand  Situatio!  0420 

DYNAMIC  FLExTbTlITY 

24-Listen/Monitor  -  Analog 

1  aoo 

a 

0.80 

0.00 

KlfiH 

Identify/Understand  Situatid  0420 

DYNAMIC  FLEXIBILITY 

SO-Read/  Digital 

1  ado 

i _ 

0.80 

0.00 

1  1478 

Identify/Understand  Situatid  0420 

DYNAMIC  FLEXiBIUTT 

31 -Scan  Digital 

f6.6o 

0.80 

0.00 

lE^ 

Identify/Understand  Situatioj  0420 

DYNAMIC  FLEXIBILITY 

32-Monitor  Digital 

fo.oo 

0.80 

0.00 

lE^ 

identify/understand  Situatioj  0430 

SPEED  OF  LIMB  MOVEMEI 

19-Read/Analog 

1  0.00 

0.80 

0.00 

1  1481 

Identiiy/Understand  Situatioj  0430 

ISPEED  OF  LIMB  MOVEMEI 

20-Scan 

S 6.66 

0.80 

0.00 

Identify/Understand  Situatio!  0430 

.SPEED  OF  LIMB  MOVEMB 

:  21 -Update/ Analog 

j  6;66 

0.80 

0.00 

Kisa 

Identify/Understand  Situatid  0430 

jSPEED  OF  LIMB  MOVEMEI 

f  22-Check  Status 

[add 

L 

0.80 

0.00 

KM 

Identify/Understand  Situatid  0430 

[SPEED  OF  LIMB  MOVEMEI 

f  23-Problem  Definition 

[  0.00 

r 

0.80 

0.00 

lEES 

Identify/Understand  Situatid  0430 

ISPEED  OF  LIMB  MOVEMEI 

1 24-Listen/Monitor  -  Analog 

1 ado 

1 

0.80 

0.00 
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Identify/Understand  Situatic^ 
]dentify/Understand  Situatioj 
Identify/Understand  Situati^ 

Identify/Understand  Situatioj 

Identify/Understand  Situatioj 
identii^Undei^tand  SItuatid 
Tdenti^/Understand  Situatioj 
Identify/Understand  Situatioj 
Identifj^Understand  Situatic^ 
identf^/Understand  Situatio; 
Identity/Understand  Situatioj 
Identify/Understand  Situatioj 
Identify/Understand  Situatid 
Identify/Understand  Situatio| 

identiiy/Understand  Situatlc^ 
jdentijy/Understand 

Identify/Understand  Situatid 
Identii^ndersterid  Situatid 
identify/Und^erstand  Situatid 

1517  iFdentil^/U^ 

Identi^/U 
Ident^/U^^^ 
Identify/Underetar^ 
identi^/Understend  Situatid^ 

Idi^tify^nderetend^l^ 

Identify/U  nd^ 

identify/Understand  Situatio; 

Identify/Understand  Situatid 
identiiy/UnderstancI  Situatioj 
Identify/Understand  Situatid 
identify/Understand  Situatid 
Ideriti^y/Un^ 

Identi^/Understand  Situatioj 

,y®MMyol®!^?®.Q?^^ 

Identiiy/Unclerstand  Situatid 
identify/Understand  Situatioj 


SPJED  OF  LIMB  MOVEME  30- 
SPEED  OF  LIMB  MOVEMEfsi- 

GROSS  BODY  EQUlLlBRly  iS 
GROSS  BO^^^ 

gross 

gross  BODY  EQuiUBRi422- 
^[GROSSBODY^E^^^ 

GROSS  b6d7eq^ 

GROSS  B0d7e^^^^^ 
TpOSSBOST^^ 

[GROSS  BODY 

GROSS 

GROSS 

GROSS  BODY  C 

gross  BODY  COO^^^^^^^ 
I^OSSBOWi^ 


IGROSS  BODY  COORDINA132 

SWIG  STRENGTH . .  jTa 

SWIC^F^G^^^^ . 1^20 

SfAtTcTfRENGfF^^^^^^  21 
^STAflCSTREN^^^^^  . [22 

Asiatic  STRE^^  [u 

STATIC  STRE^  . 30 

ISTAficSTRENG^^^^^^^  . .  ~31 

ISWICSTF^NG^  ~ 

iixPLOSi^  STT^CT^  [To 

BcSosivE 

p<PLOSi^sf  RpN^^ . 

“  EXPLOSIVE  strength . [23 

, .  ...  _ 

EXPLOSivi^^^^^^  32 

"  DYNAM  1C  19 

20 

DYNA^  21 

DYNAMIC 

ID^AMic  [23 

DYNAMIC  STR^^^^^  [24 

Idyn^^^^  30 

DYNAMICCT^^^  . 31 

IDWAMIC  STrIn^  7  32 
TRUNKS^  ”  19 

. . “20 

ITF^NK  ST  SeN^^ ^  22 

iTRUNksfREN(^ . 2Z 


‘Rea&  Digital  _ 

•Scan  Digital 
-Monitor  Digitai 
-Read/Analog  _ 

. . 

:ypd®l®/ Anal^^ _ 

-Check  Statu£^^^_^^^ 
-Problem  Definition 
7[sten/Monitor  - 
•Read/  Digitai 
•Scan  pigital 
•Monitor  Digitai 

-Read/Anal^^^^  _ _ 

■Scan . . . . . . . 

-Update/ Analog 

•Probiem  Definitipn 

:ki®?®PZ¥®M9r.iAn3!® 
-Read/  pigital 
-Scan  pig|tai 

_ 

-Read/Analog 

-Scan_ _ _ 

-Update/ Analog 
-Check  Status 
iProWem  Defin  itjon 
“Listen/Monitor  -  Analog 
-Read^pi^gital 
:S^n  Digita! 

-IWqnItor  pjgital 
-Read/Analog 

-Scan . 

-Update/  A^^^ 

-Check  S^us 
-Problem 

-Listen/Monitor  -  Analog 
-Read/  Digitai  ^ 

:M9.PJ?®Ty!.9i?®! 

-Read/Analqg 

:Scan__ 

-Update/ Analog 
-Check  Status 
-Probiem  Definition 
-Usten/Monitor  -  Analog 
-Read/  pigital 
-Scan  Digital 

-Read/Analog 

'-Scan_ . 

-Update/  Analog 

-Problem  Definition 


L  0-00  L 

1  6.00  L 

[o^oo'L 

foio’L 

['aod'r 

. oid'L 

IMl 

I  0.00  L 
i  dob !_ 
!  dob  L 

i  0.00  r 

j  0.00  L 

I  dbo  I 
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B  C 

D 

E 

F 

I 

H 

1 

1 

^te 

1 

iiiii 

Identify/Understand  Situatio 

0490 

TRUNK  STRENGTH 

24-Listen/Monitor  -  Analog 

0.00 

0.80 

0.00 

Pglil 

Identify/Understand  Situatio 

0490 

TRUNK  STRENGTH 

30-Read/  Digital 

0.00 

0.80 

0.00 

pgn 

Identify/Understand  Situatio 

0490 

TRUNK  STRENGTH 

31-Scan  Digital 

0.00 

0.80 

0.00 

pgg 

Identify/Understand  Situatio 

0490 

TRUNK  STRENGTH 

32-Monitor  Digital 

0.00 

0.80 

0,00 

iigici 

Identify/Understand  Situatio 

0500 

STAMINA 

19-Read/Analog 

0.00 

0.80 

0.00 

I  1544 

Identify/Understand  Situatio 

0500 

STAMINA 

20-Scan 

6.66 

0.80 

0.00 

Identify/Understand  Situatio 

0500 

STAMINA 

21 -Update/ Analog 

0.00 

0.80 

0.00 

Identify/Understand  Situatio 

0500 

STAMINA 

22-Check  Status 

0.00 

0.80 

0.00 

pgg 

Identify/Understand  Situatio 

0500 

STAMINA 

23-Problem  Definition 

0.00 

0.80 

0.00 

Identify/Understand  Situatio 

0500 

STAMINA 

24-Listen/Monitor  -  Analog 

o.od 

0.80 

0.00 

iEgg 

Identify/Understand  SituatIO; 

0500 

STAMINA 

30-Read/  Digital 

,0-00. 

0.80 

0.00 

Identify/Understand  Situatio 

0500 

STAMINA 

31 -Scan  Digital 

0.00 

0.80 

0.00 

lESl 

Identity/Understand  Situatid 

0500 

STAMINA 

32-Monltor  Digital 

0.00 

0.80 

0.00 

Manage  Resources 

0?10 

ORAL  COMPREHENSION 

25-Manage  Resources 

0.00 

0.80 

0.00 

Manage  Resources 

0020 

WRITTEN  COMPREHENSIC 

25-Manage  Resources 

0.00 

0.80 

0.00 

Manage  Resources 

0030 

ORAL  expression 

25-Manage  Resources 

0.00 

0.80 

0.00 

KEgi 

Manage  Resources 

0040 

25-Manage  Resources 

b.bo 

0.80 

0.00 

■  UMil 

Manage  Resources 

0050 

MEMORIZATION 

25-Manage  Resources 

0.00 

0.80 

0.00 

■BCTJ 

Manage  Resources 

0060 

PROBLEM  SENSITIVITY 

25-Manage  Resources 

0.00 

0.80 

0.00 

Manage  Resources 

0070 

pRIGINALifY 

25-Manage  Resources 

0.00 

0.80 

0.00 

Iggi 

Manage  Resources 

0080 

FLUENCY  OF  IDEAS 

25-Manage  Resources 

0.66 

0.80 

0.00 

■wan 

Manage  Resources 

0090 

FLEXIBILITY  OF  CLOSURE 

25-Manage  Resources 

6,66 

0.80 

0.00 

iggi 

Manage  Resources 

0100 

SELECTIVE  AffENT  ION 

25-Manage  Resources 

6.66 

0.80 

0.00 

Manage  Resources 

0110 

SPAT  l/i  ORIENWION 

25-Manage  Resources 

0.00 

0.80 

0.00 

■laaa 

Manage  Resources 

0120 

VISUALl^TION 

25-Manage  Resources 

^  0.00 

0.80 

0.00 

■laai 

Manage  Resources 

0130 

inductive  R^SON'NG 

25-Manage  Resources  1  0.00 

0.80 

0.00 

■taaii 

Manage  Resources 

0140 

CATEGORY  FLEXIBILITY 

25-Manage  Resources  \  0.00 

0.80 

0.00 

KE^ 

Manage  Resources 

0150 

DEDUCTIVE  REASONING 

25-Manage  Resources  !  0.00 

0.80 

0.00 

■laai 

Manage  ^ 

0160 

INFORMATION  ORDERING 

25-Manage  Resources  |  0.00 

0.80 

0.00 

MATHEMATICAL  RB^ONli 

25-Manage  Resources  i  0.00 

0.80 

0.00 

Manage  Resources  | 

1  0180  j 

NUi^ER  FAaLl^  - . . 

25-Manage  Resources 

0.00 

0.80 

0.00 

lE^ 

Manage  Res^ 

\  6m  ~  \ 

25-Manage  Resources 

0.00 

0.80 

0.00 

1  1571 

Manage  Resou^  I 

i . 

25-Manage  Resources 

0.00 

0.80 

0.00 

lE^ 

Manage  Resources 

j”'’"""ozio''^i 

IperceptuaI  speed  and 

25-Manage  Resources 

0.00 

r 

0.80 

0.00 

il^ 

Manage  Resources 

25-Manage  Resources 

0.00 

1 

0.80 

0.00 

lEifZi 

Manage  Resources 

1  0230 

25-Manage  Resources 

0.00 

r 

0.80 

0.00 

^lawi 

Manage  Resources  I 

r  0240^"^ 

INEAR  VISION 

25-Manage  Resources 

6.66 

'r~ 

! _ 

0.80 

0.00 

IKiWii 

Manage  Resources  i 

far^^^^^  . .  . . 

25-Manage  Resources 

6.66 

0.80 

0.00 

lEHl 

Manage  Resources 

0260 “ 

iNiGHTVI^ON 

25-Manage  Resources 

0.00 

\ 

0.80 

0.00 

1578 

Manage  Resources 

0270 

iNb^AfcOLORDi^RIMir^ 

25-Manage  Resources 

0.00 

1 

0.80 

0.00 

1579 

Manage  Resources 

0280 

!  PERIPHERAL  VISION 

25-Manage  Resources 

0.00 

j 

0.80 

0.00 

1580 

Manage  Resources 

0290 

DEPTH  PERCEPTION 

25-Manage  Resources 

0.00 

0.80 

0.00 

1581 

Manage  Resources 

0300 

IGLARE  SENSlfiviTY 

25-Manage  Resources 

0.00 

0.80 

0.00 

1582 

Manage  Resources 

0310 

GENERAL  HEARING 

25-Manage  Resources 

0.00 

0.80 

0.00 

1583 

Manage  Resources 

“  0^' 

AUDITORY  ATTENTION 

25-Manage  Resources 

0.00 

0.80 

0,00 

1584 

Manage  Resources 

0330 

SOUND  LOCALI^^^ 

25-Manage  Resources 

0.00 

0.80 

0.00 

1585 

Manage  Resources 

bSo 

^CONTROL  PRECISION 

25-Manage  Resources 

6.66 

r~ 

0.80 

0.00 

1586 

Manage  Resources 

0350 

iRAfE  CONTROL 

25-Manage  Resources 

0.00 

! 

0.80 

0.00 

1587 

Manage  Resources 

. 0360  " 

jWRIst-FINGER  SPEED 

25-Manage  Resources 

0.00 

0,80 

0.00 

1588 

Manage  Resources 

1  0370 

1  FINGER  DEXTERITY 

25-Manage  Resources 

6.66 

0.80 

0.00 

1589 

Manage  Resources 

1  6380 

Imanual  dexterity 

25-Manage  Resources 

6.66 

0.80 

0.00 

1590 

Manage  Resources 

1  0396 

lARM-twiD  STEADINESS^ 

25-Manage  Resources 

0.00 

0.80 

0.00 

1591 

Manage  Resources 

j  lMOO  " 

iMULTI-LIMB  COORDINATlC25-Manage  Resources 

6.66 

0.80 

0.00 

Appendix  B:  KSA  Data  Configuration  for  Treatment  Condition  #1 :  PMMMM 


B 

C 

D  I  E~ 

F 

n 

H 

1 

1 

il 

1 

;Scot^ 

■Adi' 

■Egg 

Manage  Resources 

0410 

EXTENT  FLEXIBILITY 

25-Manage  Resources 

0.00 

0.80 

0.00 

■KiKl 

Manage  Resources 

0420 

DYNAMIC  FLEXIBILITY 

25-Manage  Resources 

0.00 

0.80 

0.00 

Manage  Resources 

0430 

SPEED  OF  LIMB  MOVEMEf25-Manage  Resources 

0.00 

0.80 

0.00 

KEgg 

Manage  Resources 

0440 

GROSS  BODY  EQUILIBRlM2^Manage  Resources 

0.00 

0.80 

0.00 

KEgg 

Manage  Resources 

0450 

GROSS  BODY  c60RDINAi25-Manage  Resources 

0.00 

0.80 

0.00 

lEgg 

Manage  Resources 

0460 

STATIC  STRENGTH 

25-Manage  Resources 

0.00 

0.80 

0.00 

KEgg 

Manage  Resour^^^^  J 

0470 

EXf=»LdsiVE  STRENGTH 

25-Manage  Resources 

raiool 

0.80 

0,00 

KEfeM 

Manage  Resources 

70480'] 

DYNAMIC  STRENGTH 

25-Manage  Resources 

1  0.00  i 

0.80 

0.00 

KEgg 

Manage  Resources 

~049F1 

TRUNK  STRENGTH 

25-Manage  Resources 

[  0.00  i 

0.80 

0.00 

Manage  Resources  0500 

STAMINA 

25-Manage  Resources 

radb”; 

0.80 

0.00 

Bigg 

KEgg 

Communicate  and  Report  | 

KEtai 

Decide  and  Recommend  /  Direct 

■W»B1 

Evaluate  and  Estimate  Impact 

Identify/Understand  Situational  Picture 

KE«a 

Manage  Resources  I  | 

Appendix  C  -  Task  Time  Treatment  Condition  1  Setup  Table, 


Receive  and  Record 

29 

1 

A 

2 

1.2 

0.50 

1 

KHW 

Receive  and  Record 

29 

1 

WEM 

2 

■»gM 

1.2 

mmo^ 

0.50 

2 

HEH 

Pass  Info 

29 

2 

A 

2 

KK^9 

1.2 

fmmms^ 

KKKS 

3 

1  663 

Pass  Info 

29 

2 

A 

2 

1.2 

19.66 

19.66 

4 

Listen  Receive 

29 

3 

HH 

2 

K£K 

1.2 

■mKS 

i_ii_ 

5 

Listen  Receive 

29 

3 

A 

Kur^ 

2 

Knm 

1.2 

20.16 

20.16 

6 

fK!M 

Listen  Receive 

29 

3 

C 

2 

KDm 

1.2 

20.16 

20.16 

7 

1814 

Secondarv  Monitor 

29 

4 

mam 

KE2 

0 

■HiP:3 

1.2 

■MK2S3 

0.00 

8 

Km 

Send  Msfl 

29 

7 

A 

2 

1.2 

m^^sB 

9 

msm 

Send  Radio 

29 

7 

A 

2 

waim 

1.2 

mKiB 

10 

Km 

29 

26 

A 

KES 

2 

rnafL^im 

1.2 

1.82 

1.82 

11 

■^231 

29 

26 

KH 

KES 

2 

ma^jm 

1.2 

mmES 

1.82 

KM 

Ksn 

29 

27 

A 

1 

1.2 

■mcs^ 

miE^s 

KM 

KHK 

29 

27 

A 

■EQ] 

2 

m^Mzm 

1.2 

wmmmsssi 

3.41 

KM 

Fix  Unformatted 

29 

27 

A 

1 

TyM 

Miiawjiga 

1.2 

22.08 

22.08 

KM 

K:r:M 

Send  Overlay 

■Km 

27 

A 

2 

K:g« 

1.2 

mKSQ 

■KuSIE 

KM 

29 

27 

A 

1 

mzmm 

■HW:B 

1.2 

17 

1  883 

Send  Okiital 

29 

26 

A 

■EEH 

2 

■CT;M 

1.2 

4.34 

4.34 

18 

K:r»:K 

Decide  Action 

■Km 

11 

mam 

1 

KKK 

0.6 

■KK^ 

mm^s 

KM 

K:r»rK 

Decide  Action 

29 

11 

C 

1 

KT»7K 

Kiioa 

0.8 

0.06 

■■KEIB 

«»■ 

■nYtcH 

Decide  Action 

29 

11 

c 

1 

KEEK 

0.8 

mmiB^ 

■K^ 

■m 

Estimate  Imoact 

29 

14 

D 

1 

KMil 

0.8 

wmmEMi 

■■02^ 

Km 

Estimate  Msa  Imoact 

29 

14 

D 

1 

KSM 

Kson 

0.8 

2.55 

mmi^^ 

mfm 

KIM 

Discuss 

29 

18 

A 

2 

16 

kqm 

0.8 

12.80 

mKBEEl 

KM 

1865 

Discuss 

29 

18 

A 

2 

15 

KHM 

0.8 

12.00 

mK^s 

KM 

KtM 

Discuss 

29 

18 

A 

2 

0.8 

■m^^s 

■WtM 

Discuss 

29 

18 

C 

2 

■7«»M 

0.8 

56.67 

mm 

KrSSli 

Scan  Board 

29 

20 

B 

1 

0.8 

^■■EEO 

mmiEi^ 

KM 

nfcriM 

Scan  Mao 

29 

20 

mam 

K^ 

1 

mhym 

0.8 

3.22 

3.22 

KB 

KSM 

Uodate 

29 

Kom 

B 

1 

miiPM 

0.8 

mmEU] 

KM 

KSM 

Check  Status 

29 

22 

B 

1 

0.6 

31 

KTOM 

29 

30 

B 

1 

■aaia 

0.8 

■moEiD 

KM! 

KVi.K 

29 

31 

A 

mEQ 

1 

KEIfl 

■a^iK 

0.6 

mmKS 

mmKg 

KM 

wtr^m 

29 

32 

mam 

■EESI 

1 

0.8 

■■HE3 

0.00 

34 

InterruDt 

29 

hhk 

^■■m 

■Hmi 

HHHB 

■■m^B 

KM 

K.^-M 

Interruot 

29 

mmiiii 

IMIIIIII 

imHHi 

■mmmi 

KM' 

Kf^K 

COR  ^in 

29 

2 

hhki 

HHKi 

mmmii 

HHIKi 

KM 

Km 

interruot 

29 

■r>!tM 

KRiK 

khhhi 

■■mm 

■■■■1 

mi^^H 

KM 

K» 

Icon  Mvt/Color  Cho 

29 

j^miii 

HHHim 

mmmH 

■■■■■ 

■mi^^ 

■cum 

KTiM 

Interruot  CMD 

mrm 

iHKm 

mmsHi 

■mmm 

■■KK 

KM 

Interrupt  CMD 

29 

iw/yyj] 

■■mi 

■■mm 

^^mm 

41 

Appendix  D  -  Data  From  Simulation  Runs. 
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Data  From  Fractional  Factorial-  KSA  Runs. 


Random  #  Seed  =  1.0 
T1-KSA-PMMMM 

Battalion 

Commander 

D 

Adj 

# 

Taskload 

(Workload) 

(Util) 

a 

B 

31.99 

B 

B 

31.88 

Replication 

3 

3 

31.95 

Replication 

4 

4 

31.88 

Replication 

B 

B 

31.68 

Replication 

B 

B 

Replication 

B 

B 

Replication 

B 

B 

Replication 

B 

B 

31.92 

Replication 

10 

7 

31.99 

Replication 

m 

B 

31.92 

Replication 

B 

31.96 

Replication 

13 

10 

31.77 

m 

ID 

31.81 

m 

31.84 

Replication 

m 

B 

31.80 

Replication 

m 

B 

Replication 

18 

14 

31.88 

Replication 

B 

32.00 

Replication 

E*l 

B 

Average 

31.88 

Random#  Seed  =  1.0 
T3-KSA-PPMPM 

Battalion 

M 

Adj 

Workload 

Commander 

n 

# 

(Util) 

1 

D 

46.30 

B 

B 

46.22 

Replication 

B 

B 

46.29 

Replication 

4 

4 

46.25 

Replication 

B 

B 

45.97 

Replication 

agi 

B 

Replication 

B 

B 

Replication 

B 

B 

Replication 

B 

B 

46.31 

Replication 

B 

B 

46.36 

Replication 

B 

B 

46.31 

B 

B 

■ESEEHI 

Replication 

B 

lltM 

46.18 

Replication 

B 

B 

46.21 

Replication 

B 

B 

46.20 

Replication 

B 

B 

46.24 

Replication 

EB 

B 

Replication 

KFl 

B 

46.29 

Replication 

IFl 

B 

46.42 

Replication 

Kil 

D 

B 

46.26 

Random  #Seed  =  1.01 


IT2-KSA-MPPMM 


Battalion 

Commander 

D 

Adj 

# 

Workload 

(Util) 

Replication 

B 

1 

Replication 

B 

B 

32.66 

Replication 

B 

B 

32.76 

Replication 

B 

B 

32.62 

Replication 

B 

B 

32.40 

Replication 

B 

B 

Replication 

B 

B 

Replication 

B 

B 

Replication 

B 

B 

32.67 

Replication 

Kill 

B 

32.79 

Replication 

B 

B 

32.65 

Replication 

B 

B 

32.73 

Replication 

B 

B 

32.40 

Replication 

14 

11 

32.48 

Replication 

B 

B 

32.59 

Replication 

B 

B 

32.42 

Replication 

B 

B 

Replication 

B 

B 

32.60 

Replication 

19 

15 

32.75 

Replication 

B 

B 

z: 

32.62 

Random#  Seed  =  1.0 
T4-KSA-MMPPM 

Battalion 

Commander 

D 

Adj 

# 

Workload 

(Util) 

Replication 

B 

B 

47.15 

Replication 

B 

B 

46.99 

B 

B 

47.09 

Replication 

B 

B 

46.98 

Replication 

B 

B 

46.68 

Replication 

B 

B 

Replication 

B 

B 

Replication 

B 

B 

Replication 

B 

B 

47.05 

Replication 

B 

B 

47.15 

Replication 

EQ 

B 

47.03 

Replication 

B 

B 

■EEim 

Replication 

B 

Kill 

46.80 

Replication 

Kgl 

m 

46.87 

Replication 

B 

B 

46.94 

Replication 

B 

■o 

46.85 

Replication 

ra 

B 

Replication 

B 

Kgl 

47.00 

Replication 

B 

47.17 

Replication 

ws% 

B 

Average 

15 

z: 

46.99 
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Data  From  Fractional  Factorial-  Task  Time  Runs. 


Random  #  Seed  =  1.0 


TI-TTime-PMMMM 


Battalion 

Commander 


Replication 


Replication 


Replication 


Replication 


Replication 


Replication 


Replication 


QSillSBBiiElEIil 


Replication 


Replication 


Replication 


Replication 


Replication 


lication 


Replication 


Replication 


Replication 


mm 


Average 


Utilization 

(Util) 


39.28 

0.9250 

49.33 

90.33 

10.47 

9.60 

Random  #  Seed  s  1.0 


T2-TTlme-MPPMM 


Battalion 

Commander 


ReDlication  2 


3 


Reolication 


Reolication 


Reolication 


i  •li  W:1  ffflTHBI 


Reolication 


QilS^QS&IH 


Reolication 


Reolication 


Replication 


Replication 


Replication 


Replication 


Replication 


Replication 


[aS^SSQSiHII 


Replication 


Replication 


Replication 


Replication 


Replication 


_ 

_ [I^^EBI 

_ 

IBiTaWgl 

1 1 1  III  I  III  liMI  1 1  III  HI  I 


Average 

1  15 

_ 

1  0.93  1 

_ 47.13  1 

1  89.07  1 

1  10.33 

Random  #  Seed  =  1.0 


T3-TTime-PPMPM 


Battalion 

Commander 

# 

1 

Workioad 

(Util) 

Replication 

T 

1 

38.73 

Replication 

B 

B 

38.36 

Replication 

3 

3 

38.51 

D 

B 

38.58 

B 

B 

38.51 

Replication 

6 

6 

38.71 

Replication 

B 

B 

38.59 

Replication 

B 

B 

38.60 

Replication 

B 

B 

38.64 

Replication 

Da 

BRl 

38.66 

Replication 

ni 

B 

BBDBI 

Replication 

m 

ED 

38.57 

Replication 

EEl 

ED 

38.69 

Replication 

EEl 

38.53 

Replication 

15 

14 

38.37 

Replication 

m 

B 

Replication 

m 

X 

HBDI 

Replication 

18 

15 

38.76  1 

Replication 

BRl 

B 

mmam\ 

Replication 

20 

n 

^^Bll 

Average 

Ba 

38.59  1 

Number  of 

Tasks 

Tasks 

Tasks 

Queues 

interrupted 

Suspended 

Dropped 

(Opdata) 

(Opdata) 

(Opdata) 

(Opdata) 

Utilization 

(Utii) 


4 
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Random#  Seed-  1.0 


T4-TTime-MMPPM 


* 
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Random  #  Seed  =  1.0 


T5-TTime-MPMMP 


Random  #  Seed  =  1.0 


T6-TTime-PMPMP 


Reolication 


Reolication 


Reolicatton 


Replication 


Workioad 

(Util) 


39.57 


39.44 


39.52 


39.43 


39.18 


Replication 


Replication 


Reolicatton 


Reolicatton 


Replication 


Repiicatton 


Reolication 


Repiication 


Replication 


Replication 


Averaoe 


\mm\ 

39.44  I 

I  0.9244  I 

I  50.60  I 

I  90.27  I 

I  10.80  I 

I  10.93 

Random# Seeds  i.o  Treatment  Level  =  20%  T7-TTime-MMMPP 


Random  #  Seed  =  1.0 


T8-TTime-PPPPP 


* 


miMgi 

■IM9I 

HgMffijl 

EH 

% 
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Data  From  Center  Point  Runs. 

T  reatment  #  9  Random  #  Seed  = 


1.0000 


Workload  Utilization 

(Util)  (Util) 


39.18 


39.49 

39.57 


39.48 

39.53 

39.29 

39.34 


39.39 

39.33 


39.44 

39.58 


39.44 

0.9244 

50.60  1 

1  90.27  1 

1  10.80  1 

1  10.93 

Random  #  Seed  =  2.00 


Workload 

(Util) 

39.41 


39.43 


39.45 

39.51 


39.62 

39.29 

39.06 

39.36 


39.50 

39.52 

39.60 

39.30 

39.15 

39.29 


39.39 


39.39  I  0.9240  I  49.87  I  90.47  I  10.60  I  11.13 


Battalion 

Commander 


Replication 


Replication 


Replication 


Replication 


Replication 


Replication 


Replication 


Replication 


Replication 


Replication 


Replication 


Replication 


Replication 


Replication 


Replication 


Replication 


Replication 


mm 


Average 


39.41  I 

I  0.9246  I 

r  50.93  I 

I  89.60  I 

I  10.73  1 

Treatment#  14 _  Random  #  Seed  =  6 


Battalion  Adj 

Commander  # 


Replication 

Replication 

Replication 

Replication 

Replication 

Replication 


Replication 

Replication 

Replication 

Replication 

Replication 

Replication 

Replication 


Bill 

EDO 

mwa 

■EIO 

BUM 

mm 


Replication 

Replication 

Replication 


■niKFi 

laiEi 

mm 


Worktoad 

(Util) 


39.31 

39.24 

39.26 

39.30 


39.43 

39.52 


39.26 


39.59 

39.44 

39.26 

39.66 

39.32 


39.09 

39.31 

39.37 


0.9244  1 

1  49.67  1 

L  89.60  1 

1  10.80  1 

1  10.93 

Average 


Treatment#  16 


Battalion 

Commander 


Reolication 


ReDlication 


Reolication 


Reolication 


Random  #  Seed  = 


Adj 

# 

Workload 

(Util) 

Utilization 

(Util) 

Number  of  Tasks  Tasks 

Queues  Interrupted  Suspended 
(Opdata)  (Opdata)  (Oodata) 

Reolication 


Reolication 


Reolication 


Reolication 


Reolication 


lication 


lication 


Reolication 


Reolication 


Reolication 


Reolication 


tamm 


139 


Treatment#  17 


Battalion 

Commander 

D 

1 

Workload 

(Util) 

n 

B 

KEQB 

B 

MM 

BQ^B 

[2SsIBSIS!SHi 

B 

B 

39.32 

I£9S!S!!^H 

4 

B 

39.56 

[^SSSISSIiH 

B 

D 

39.42 

1  Replication 

6 

X 

■I^HI 

7 

B 

39.41 

1  Replication 

B 

B 

39.45 

B 

B 

39.29 

[^Baai 

Kbl 

■1 

Replication 

11 

X 

Replication 

m 

■1 

BBBHI 

EQ 

B 

39.36 

EQ 

B 

39.59 

Ea 

■1 

■■BBII 

[^3^IS!3Hi 

m 

EQ 

39.15  I 

EQ 

EQ 

hsbi 

JSSSSSMSMM 

EQ 

EQ 

45.72  I 

1  Replication 

EQ 

■1 

wst 

EQ 

39.27  I 

Replication 

m 

■1 

■■■■I 

Replication 

EQ 

39.25  I 

wst 

B 

■■■■I 

EQ 

39.41  1 

m 

■1 

BHBI 

[Average 

EQ 

H 

40.21  1 

Random  #  Seed  = 

Utilization 
(Util) 


Number  of 

Queues 
data' 


Tasks 
Interrupted 
3data) 
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Appendix  E  -  Multicollinearity  Tests. 


6 


f 


S  AS  Program  -  PROC  CORK. 


options  pageno=l  f onndlim= ' - ' ; 

TITLE  'CoHOST  1/4  Replicate  of  2**5  Design*; 

EATA  cohost; 

INPUT  car  drd  eei  usp  mrs  subj  tltt  tlksa  util  noque  tskint  tsksup  tskdrp; 

*  Independent  variables  -  Factors:; 


Factor 

Factor 

Factor 

Factor 

Factor 


CAR  ~  Communicate  and  Report; 

DRD  -  Decide  and  Recommend  /  Direct; 

EEI  -  Evaluate  and  Estimate  Impact; 

USP  -  Identify  /  Understand  Situational  Picture; 
MRS  -  Manage  Resources; 


Simulated  subjects:  subj; 


*  Dependent  variables:; 

*  TLTT  -  Taskload  from  Workload  in  the  Task  Time  runs; 

*  TLKSA  -  Taskload  from  Workload  in  the  KSA  /  JASS  data  runs; 

*  UTIL  -  %  Utilization  over  run; 

*  NOQUE  -  Number  of  gueues  generated  during  the  run,  ; 

*  i.e.,  the  number  of  times  a  task  was  queued  up; 

*  TSKINT  "  Number  of  times  a  task  was  interrupted; 

*  TSKSUP  -  Number  of  times  a  task  was  suspended ; 

*  TSKDRP  -  Nimnber  of  times  a  task  was  dropped; 


*  Provide  input  data  as  CARDS  statements  ; 

*  1/4  replicate  portion:  ; 

*  5  Factors  -  2  levels/ factor^  32  Treatment  Combinations  x  ; 

*  1  subject  per  cell  = 

*  32  Treatments  if  full  factorial; 

*  8  Treatments  for  1/4  fractional  factorial; 

*  Central  composite  center  point  treatment:  ; 

*  11  treatments  on  center  point  to  provide  3rd  treatment  level; 


CARDS; 


1 

-1 

-1 

-1 

-1 

1 

39.275 

31.884 

0.9250 

49.33 

90.33 

10.47 

9.60 

-1 

1 

1 

-1 

-1 

1 

40.084 

32.625 

0.9256 

47.13 

89.07 

10.33 

9.40 

1 

1 

-1 

1 

1 

38.588 

46.259 

0.9213 

55.13 

88.47 

10.47 

14.20 

-1 

-1 

1 

1 

1 

36.793 

46.988 

0.9242 

51.73 

88.47 

10.67 

12,20 

-1 

1 

-1 

-1 

1 

1 

39.810 

31.557 

0.9242 

42.80 

90.87 

10.40 

8.93 

1 

-1 

1 

-1 

1 

1 

39.439 

32.952 

0.9244 

50.60 

90.27 

10.80 

10.93 

-1 

-1 

-1 

1 

1 

1 

39.217 

45.920 

0.9240 

48.27 

89.07 

10.40 

11.07 

1 

1 

1 

1 

1 

1 

38.891 

44.180 

0.9183 

58.47 

87.27 

9.47 

15.33 

0 

0 

0 

0 

0 

1 

39.439 

39.439 

0.9244 

50.60 

90.27 

10.80 

10.93 

0 

0 

0 

0 

0 

2 

39.391 

39.391 

0.9240 

49.87 

90.47 

10.60 

11.13 

0 

0 

0 

0 

0 

3 

39.372 

39.372 

0.9242 

49.87 

89.87 

10.80 

11.13 

0 

0 

0 

0 

0 

4 

39.351 

39.351 

0.9243 

49.53 

90.13 

10.93 

10.53 

0 

0 

0 

0 

0 

5 

39.407 

39.407 

0.9246 

50.93 

89.60 

10.73 

10.47 

0 

0 

0 

0 

0 

6 

39.367 

39.367 

0.9244 

49.67 

89.60 

10.80 

10.93 

0 

0 

0 

0 

0 

7 

39.320 

39.320 

0.9246 

49.07 

90.27 

10.73 

10.93 

0 

0 

0 

0 

0 

8 

39.346 

39.346 

0.9239 

49.73 

89.87 

10.87 

10.60 

0 

0 

0 

0 

0 

9 

40.207 

40.207 

0.9235 

50.00 

89.87 

10,80 

10,87 

0 

0 

0 

0 

0 

10 

39.340 

39.340 

0.9238 

50.07 

89.67 

10.73 

11.20 

0 

0 

0 

0 

0 

11 

39.382 

39.382 

0.9246 

50.40 

89.80 

10.60 

10.80 

*  Regression  Runs:  ; 

*  First,  test  for  any  multicollinearity  problems: 
options  pageno=l  fonndlim= ' - ' ; 

*  Generate  a  full  correlation  table:  ; 


proc  corr  data=cohost; 

TITLE  'CoHOST  Dependent  Variable  Correlation  Matrix:  Multicollinearity  Test'; 

var  car  drd  eei  usp  mrs  tltt  tlksa  util  noque  tskint  tsksup  tskdrp; 

run; 
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S  AS  Output  -  PROC  CORR, 


y 


4 


CoHOST  Dependent  Variable  Correlation  Matrix 


Multicollinearity  Test  49 

13:05  Sunday^  June  24,  2001 


The  CORR  Procedure 


\  Variables: 

car 

tskint 

drd  eei  usp  mrs  tltt 

tsksup  tskdrp 

Simple  Statistics 

tlksa 

Util  noque 

Variable 

N 

Mean 

Std  Dev 

Sum 

Minimum 

Maximum 

car 

19 

0 

0.66667 

0 

-1.00000 

1 .00000 

drd 

19 

0 

0,66667 

0 

-1.00000 

1 .00000 

eei 

19 

0 

0.66667 

0 

-1.00000 

1 .00000 

usp 

19 

0 

0.66667 

0 

-1.00000 

1 .00000 

mrs 

19 

0 

0.66667 

0 

-1.00000 

1 .00000 

tltt 

19 

39.26416 

0.69607 

746.01900 

36.79300 

40.20700 

tlksa 

19 

39.27826 

4.56953 

746.28700 

31.55700 

46.98800 

util 

19 

0.92386 

0.00158 

17.55330 

0.91830 

0.92560 

noque 

19 

50.16842 

3.04669 

953.20000 

42.80000 

58.47000 

tskint 

19 

89.64421 

0.85963 

1703 

87.27000 

90.87000 

tsksup 

19 

10.60000 

0.32498 

201 .40000 

9.47000 

10.93000 

tskdrp 

19 

11.11474 

1 .48699 

211.18000 

8.93000 

15,33000 

Pearson  Correlation  Coefficients,  N  =  19 

Prob  > 

|r|  under  HO: 

Rho=0 

car 

drd 

eei 

usp 

mrs 

tltt 

car 

1 .00000 

0.00000 

0.00000 

0.00000 

0.00000 

0.03460 

1.0000 

1.0000 

1.0000 

1.0000 

0.8882 

drd 

0.00000 

1 .00000 

0.00000 

0.00000 

0.00000 

0.31714 

1.0000 

1.0000 

1.0000 

1.0000 

0.1858 

eei 

0.00000 

0.00000 

1 .00000 

0.00000 

0.00000 

-0.20149 

1.0000 

1,0000 

1 .0000 

1.0000 

0.4081 

usp 

0.00000 

0.00000 

0.00000 

1.00000 

0.00000 

-0.61284 

1.0000 

1.0000 

1.0000 

1.0000 

0.0053 

mrs 

0.00000 

0.00000 

0.00000 

0.00000 

1 .00000 

0.31331 

1.0000 

1.0000 

1.0000 

1 .0000 

0.1915 

tltt 

0.03460 

0.31714 

-0.20149 

-0.61284 

0.31331 

1 .00000 

0.8882 

0.1858 

0.4081 

0.0053 

0.1915 

tlksa 

-0.03310 

-0.05695 

0.02052 

0.99078 

-0.05739 

-0.62500 

0.8930 

0.8169 

0.9336 

<.0001 

0.8155 

0.0042 

util 

■0.47420 

-0.43205 

-0.10538 

-0.60066 

-0.27398 

0.24293 

0.0402 

0.0647 

0.6677 

0.0065 

0.2563 

0.3163 

noque 

0.64551 

0.09847 

0.33917 

0.64934 

-0.08698 

-0.44265 

0.0028 

0.6884 

0.1555 

0.0026 

0.7233 

0.0577 

tskint 

-0.11051 

-0.23848 

-0.35480 

-0.70379 

0.11051 

0.50882 

0.6524 

0.3255 

0.1361 

0.0008 

0.6524 

0.0261 

tsksup 

-0.15129 

-0.42823 

-0.12052 

-0.25386 

-0.22309 

0.09853 

0.5364 

0.0674 

0.6231 

0.2943 

0.3586 

0.6882 

tskdrp 

0.47411 

0.22753 

0.22753 

0.78122 

0.04820 

-0.50938 

0.0403 

0.3489 

0.3489 

<.0001 

0.8447 

0.0259 
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Pearson  Correlation  Coefficients,  N  =  19 

HO:  Rho=0 


car 

tlksa 

-0.03310 

drd 

0.8930 

>0.05695 

eei 

0.8169 

0.02052 

usp 

0.9336 

0.99078 

mrs 

<.0001 

-0,05739 

tltt 

0.8155 

-0.62500 

tlksa 

0.0042 

1 .00000 

util 

-0.51917 

noque 

0.0227 

0.61908 

tskint 

0.0047 

-0.66283 

tsksup 

0.0020 

-0.14484 

tskdrp 

0.5541 

0.73953 

0.0003 

Prob  > 

|rl  under 

util 

noque 

-0.47420 

0.64551 

0.0402 

0.0028 

-0.43205 

0.09847 

0.0647 

0.6884 

-0.10538 

0.33917 

0.6677 

0.1555 

>0.60066 

0.64934 

0.0065 

0.0026 

-0.27398 

>0.08698 

0.2563 

0.7233 

0.24293 

>0.44265 

0.3163 

0.0577 

-0.51917 

0.61908 

0.0227 

0.0047 

1 .00000 

-0.77935 

<.0001 

>0.77935 

<.0001 

1 .00000 

0.71330 

-0.74302 

0.0006 

0.0003 

0.71186 

>0.42306 

0.0006 

0,0711 

-0.89196 

0.91791 

<.0001 

<.0001 

tskint 

tsksup 

>0.11051 

-0.15129 

0.6524 

0.5364 

>0.23848 

-0.42823 

0.3255 

0.0674 

-0.35480 

-0.12052 

0.1361 

0.6231 

-0.70379 

-0.25386 

0.0008 

0.2943 

0.11051 

-0.22309 

0.6524 

0.3586 

0.50882 

0.09853 

0.0261 

0.6882 

>0,66283 

-0.14484 

0.0020 

0.5541 

0.71330 

0,71186 

0.0006 

0.0006 

-0.74302 

-0.42306 

0.0003 

0.0711 

1.00000 

0.66042 

0.0021 

0.66042 

0.0021 

1 .00000 

>0.80005 

-0.52516 

<.0001 

0.0210 

tskdrp 
0.47411 
0.0403 
0.22753 
0.3489 
0.22753 
0.3489 
0.78122 
<.0001 
0.04820 
0.8447 
>0.50938 
0.0259 
0.73953 
0.0003 
>0.89196 
<.0001 
0.91791 
<.0001 
-0.80005 
<.0001 
>0.52516 
0.0210 
1 .00000 


Appendix  F  -  Determination  Of  Multiple  Linear  Regression  Models  For  Standardized  Data. 
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SAS  PROC  REG  for  MLR  Model  Run. 


The  multiple  linear  regression  model  to  be  used  in  this  thesis  is  determined  by  SAS  runs 
that  generate  the  regression  parameter  estimates  (beta  weights)  for  each  dependent  variable.  The 
MRS  independent  variable  for  each  dependent  variable  is  examined  to  identify  the  smallest  p 
value.  This  value  is  then  used  as  a  threshold  to  determine  which  of  the  other  coefficients  are 
significant  and  whose  value  should  be  included  in  the  regression  model  for  that  dependent 
variable. 


options  pageno=l  f  ormdliin= '  - '  ; 

TITLE  ‘CoHOST  1/4  Replicate  of  2**5  Design'; 

DHTA  cohost; 

INPUT  car  drd  eei  usp  mrs  subj  tltt  tlksa  util  nogue  tskint  tsksup  tskdrp; 

*  Independent  variables  -  Factors:; 

*  Factor  A:  CAR  -  Communicate  and  Report; 

*  Factor  B:  DRD  -  Decide  and  Recommend  /  Direct; 

*  Factor  C:  EEI  -  Evaluate  and  Estimate  Impact; 

*  Factor  D:  USP  -  Identify  /  Understand  Situational  Picture; 

*  Factor  E:  MRS  -  Manage  Resources; 

*  Simulated  subjects:  subj ; 


*  Dependent  variables:; 

*  TLTT  -  Taskload  from  Workload  in  the  Task  Time  runs; 

*  TLKSA  -  Taskload  from  Workload  in  the  KSA  /  JASS  data  runs; 

*  UTIL  -  %  Utilization  over  run; 

*  NOQUE  -  Number  of  gueues  generated  during  the  run,  ; 

*  i.e.,  the  number  of  times  a  task  was  queued  up; 

*  TSKINT  -  Number  of  times  a  task  v/as  interrupted; 

*  TSKSUP  -  Number  of  times  a  task  v/as  suspended; 

*  TSKDRP  “  Number  of  times  a  task  was  dropped; 

*  Provide  input  data  as  CARDS  statements  ; 

*  1/4  replicate  portion: 

*  5  Factors  -  2  levels/f actor=  32  Treatment  Combinations  x  ; 

*  1  subject  per  cell  = 

*  32  Treatments  if  full  factorial; 

*  8  Treatments  for  1/4  fractional  factorial; 

*  Central  composite  center  point  treatment:  ; 

*  11  treatments  on  center  point  to  provide  3rd  treatment  level; 

*  . 

CARDS; 


1 

-1 

-1 

-1 

-1 

1 

39.275 

31.884 

0.9250 

49.33 

90.33 

10.47 

9,60 

-1 

1 

1 

-1 

-1 

1 

40.084 

32.625 

0.9256 

47.13 

89.07 

10.33 

9.40 

1 

1 

-1 

1 

-1 

1 

38.588 

46.259 

0.9213 

55.13 

88.47 

10.47 

14.20 

-1 

-1 

1 

1 

-1 

1 

36.793 

46.988 

0.9242 

51.73 

88.47 

10.67 

12.20 

-1 

1 

-1 

-1 

1 

1 

39.810 

31.557 

0.9242 

42.80 

90.87 

10.40 

8.93 

1 

-1 

1 

-1 

1 

1 

39.439 

32.952 

0.9244 

50.60 

90.27 

10.80 

10.93 

-1 

-1 

-1 

1 

1 

1 

39.217 

45.920 

0.9240 

48.27 

89.07 

10.40 

11.07 

1 

1 

1 

1 

1 

1 

38.891 

44.180 

0.9183 

58.47 

87.27 

9.47 

15.33 

0 

0 

0 

0 

0 

1 

39.439 

39.439 

0.9244 

50.60 

90.27 

10.80 

10,93 

0 

0 

0 

0 

0 

2 

39.391 

39.391 

0.9240 

49.87 

90.47 

10.60 

11.13 

0 

0 

0 

0 

0 

3 

39.372 

39.372 

0.9242 

49.87 

89.87 

10.80 

11.13 

0 

0 

0 

0 

0 

4 

39.351 

39.351 

0.9243 

49.53 

90.13 

10.93 

10.53 

0 

0 

0 

0 

0 

5 

39.407 

39.407 

0.9246 

50.93 

89.60 

10.73 

10.47 

0 

0 

0 

0 

0 

6 

39.367 

39.367 

0.9244 

49.67 

89.60 

10.80 

10.93 

0 

0 

0 

0 

0 

7 

39.320 

39.320 

0.9246 

49.07 

90.27 

10.73 

10.93 

0 

0 

0 

0 

0 

8 

39.346 

39.346 

0.9239 

49.73 

89.87 

10.87 

10.60 

0 

0 

0 

0 

0 

9 

40.207 

40.207 

0.9235 

50.00 

89.87 

10.80 

10.87 

0 

0 

0 

0 

0 

10 

39.340 

39.340 

0.9238 

50.07 

89.67 

10.73 

11.20 

0 

0 

0 

0 

0 

11 

39.382 

39.382 

0.9246 

50.40 

89.80 

10.60 

10.80 
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options  pageno=l  fonndliiti=’ -  ' ; 

*  Use  straight  proc  reg  on  standardized  data  to  build  the  regression  models  ; 

*  Standardize  the  data  and  rerun; 
options  pageno=l  fontidlim= '  -  ' ; 

proc  standard  mean=0  std=l  out=cohosts  data=cohost; 

var  tltt  tlksa  util  noque  t skint  tsksup  tskdrp; 


proc  rag  corr  data=cohosts ; 

TITLE  'CoHOST  Multiple  Linear  Regression  Model/  Standardized  Data-  Variable  TLTT'; 
model  tltt  =  car  drd  eei  usp  mrs; 


proc  rag  corr  data^cohosts ; 

TITLE  ‘CoHOST  Multiple  Linear  Regression  Model/ 
model  tlksa  =  car  drd  eei  usp  mrs; 

run; 

proc  rag  corr  data=:cohosts ; 

title  'CoHOST  Multiple  Linear  Regression  Model/ 
model  util  =  car  drd  eei  usp  mrs; 

run; 

proc  rag  corr  data=cohosts; 

TITLE  'CoHOST  Multiple  Linear  Regression  Model/ 
model  noque  =  car  drd  eei  usp  mrs; 

run; 

proc  rag  corr  data=cohosts ; 

TITLE  'CoHOST  Multiple  Linear  Regression  Model/ 
model  tskint  =  car  drd  eei  usp  mrs 


Standardized  Data-  Variable  TLKSA' 


Standardized  Data-  Variable  UTIL'; 


Standardized  Data-  Variable  NOQUE' 


Standardized  Data-  Variable  TSKINT 


run; 

proc  rag  corr  data=cohosts ; 

TITLE  'CoHOST  Multiple  Linear  Regression  Model/  Standardized  Data-  Variable  TSKSUP 
model  tsksup  =  car  drd  eei  usp  mrs; 


run; 

proc  rag  corr  data=cohosts; 

TITLE  'CoHOST  Multiple  Linear  Regression  Model/  Standardized  Data-  Variable  TSKDRP 
model  tskdrp  =  car  drd  eei  usp  mrs; 

run; 


4 


148 


S  AS  Output  -  PROC  REG. 


CoHOST  Multiple  Linear  Regression  Model/  Standardized  Data-  Variable  TLTT  1 


11:07  Thursday,  July  5,  2001 


The  REG  Procedure 


Correlation 


Variable 

car 

drd 

eei 

usp 

mrs 

tltt 

car 

1.0000 

0.0000 

0.0000 

0.0000 

0.0000 

0.0346 

drd 

0.0000 

1.0000 

0.0000 

0.0000 

0.0000 

0.3171 

eei 

0.0000 

0.0000 

1 .0000 

0.0000 

0.0000 

-0.2015 

usp 

0.0000 

0.0000 

0.0000 

1.0000 

0.0000 

-0.6128 

mrs 

0.0000 

0.0000 

0.0000 

0.0000 

1.0000 

0.3133 

tltt 

0.0346 

0.3171 

-0.2015 

-0.6128 

0.3133 

1 .0000 

CoHOST  Multiple 

Linear  Regression  Model/  Standardized  Data- 

Variable  TLTT 

2 

The  REG  Procedure 


11:07  Thursday,  July  5,  2001 


Model:  M0DEL1 


Dependent  Variable:  tltt 


Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

5 

11.08997 

2.21799 

4.17 

0.0176 

Error 

13 

6.91003 

0.53154 

Corrected  Total 

18 

18.00000 

Root  MSE 

0.72907 

R-Square 

0.6161 

Dependent 

Mean 

-8.0637E-15 

Adj  R-Sq 

0.4685 

Coeff  Var 

-9.04133E15 

Parataeter  Estimates 


Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  |t| 

Intercept 

1 

-8.0637E-15 

0.16726 

-0.00 

1.0000 

car 

1 

0.05190 

0.25776 

0.20 

0.8435 

drd 

1 

0,47571 

0.25776 

1.85 

0.0879 

eei 

1 

-0.30223 

0.25776 

-1.17 

0.2620 

usp 

1 

-0.91927 

0.25776 

-3.57 

0.0034 

mrs 

1 

0.46996 

0.25776 

1.82 

0.0913 

CoHOST  Multiple  Linear  Regression  Model/  Standardized  Data-  Variable  TLKSA 


The  REG  Procedure 
Correlation 


11:07  Thursday,  July  5, 


3 

2001 


Variable 

car 

drd 

eei 

usp 

mrs 

tlksa 

car 

1.0000 

0.0000 

0.0000 

0.0000 

0.0000 

-0.0331 

drd 

0.0000 

1.0000 

0.0000 

0.0000 

0.0000 

-0.0570 

eei 

0.0000 

0.0000 

1 .0000 

0.0000 

0.0000 

0,0205 

usp 

0.0000 

0.0000 

0.0000 

1 .0000 

0.0000 

0.9908 

mrs 

0.0000 

0.0000 

0.0000 

0.0000 

1.0000 

-0.0574 

tlksa 

-0.0331 

-0.0570 

0.0205 

0.9908 

-0.0574 

1 .0000 

Cohost  Multiple 

Linear  Regression  Model/  Standardized  Data- 

Variable  TLKSA 

4 

The  REG  Procedure 


11:07  Thursday,  July  5,  2001 


Model:  M0DEL1 


Dependent  Variable:  tlksa 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF  Squares 

Square  F 

Value  Pr 

Model 

5  17.81469 

3.56294 

249.95  <.( 

Error 

13  0.18531 

( 

0.01425 

Corrected 

Total 

18  18.00000 

Root 

MSE 

0.11939 

R- Square 

0.9897 

Dependent 

Mean  -1.1931E-15 

Adj  1 

R-Sq 

0.9857 

Coeft 

‘  Var 

-1.00067E16 

Parameter  Estimates 

Parameter  Standard 

Variable 

DF 

Estimate 

Error 

t 

Value 

Pr  >  |t| 

Intercept 

1 

-1.1931E-15 

0.02739 

-0.00 

1 .0000 

car 

1 

•0.04965 

0.04221 

-1.18 

0.2606 

drd 

1 

-0.08543 

0.04221 

-2.02 

0.0640 

eei 

1 

0.03077 

0.04221 

0.73 

0.4789 

usp 

1 

1.48617 

0.04221 

35.21 

<.0001 

mrs 

1 

•0.08609 

0.04221 

-2.04 

0.0623 

4 
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CoHOST  Multiple  Linear  Regression  Model/  Standardized  Data-  Variable  UTIL  5 


The  REG  Procedure 
Correlation 


11:07  Thursday,  July  5,  2001 


Variable 

car 

drd 

eei 

usp 

mrs 

util 

car 

1.0000 

0.0000 

0.0000 

0.0000 

0.0000 

-0.4742 

drd 

0.0000 

1.0000 

0.0000 

0.0000 

0.0000 

-0.4321 

eei 

0.0000 

0.0000 

1.0000 

0.0000 

0.0000 

-0.1054 

usp 

0.0000 

0.0000 

0.0000 

1.0000 

0.0000 

-0.6007 

mrs 

0,0000 

0.0000 

0.0000 

0.0000 

1 .0000 

-0.2740 

util 

-0.4742 

-0.4321 

-0.1054 

-0.6007 

-0.2740 

1 .0000 

CoHOST  Multiple 

Linear  Regression  Model/  Standardized  Data- 

Variable  UTIL 

6 

11:07  Thursday,  July  5,  2001 


The  REG  Procedure 


Model:  M0DEL1 
Dependent  Variable:  util 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

5 

15.45296 

3.09059 

15.77 

<.0001 

Error 

13 

2.54704 

0.19593 

Corrected  Total 

18 

18.00000 

Root  MSE 

0.44264 

R- Square 

0.8585 

Dependent 

Mean 

-1.3302E-13 

Adj  R-Sq 

0.8041 

Coeff  Var 

-3.32753E14 

Parameter  Estimates 


Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  |t| 

Intercept 

1 

-1.3302E-13 

0.10155 

-0.00 

1.0000 

car 

1 

-0.71130 

0.15650 

-4.55 

0.0005 

drd 

1 

-0.64808 

0.15650 

-4.14 

0.0012 

eei 

1 

-0.15807 

0.15650 

-1,01 

0.3309 

usp 

1 

-0.90099 

0.15650 

-5.76 

<.0001 

mrs 

1 

-0.41098 

0.15650 

-2.63 

0.0209 

CoHOST  Multiple  Linear  Regression  Model/  Standardized  Data-  Variable  NOQUE  7 


11:07  Thursday,  July  5,  2001 


The  REG  Procedure 


Correlation 


Variable 

car 

drd 

eei 

usp 

mrs 

noque 

car 

1.0000 

0.0000 

0.0000 

0.0000 

0.0000 

0.6455 

drd 

0.0000 

1.0000 

0.0000 

0.0000 

0.0000 

0.0985 

eei 

0.0000 

0.0000 

1.0000 

0.0000 

0.0000 

0.3392 

usp 

0.0000 

0.0000 

0.0000 

1.0000 

0,0000 

0,6493 

mrs 

0.0000 

0.0000 

0.0000 

0.0000 

1.0000 

-0.0870 

noque 

0.6455 

0.0985 

0.3392 

0.6493 

-0.0870 

1 .0000 

CoHOST  Multiple 

Linear  Regression  Model/  Standardized  Data- 

Variable  NOQUE 

8 

11:07  Thursday,  July  5,  2001 


The  REG  Procedure 


Model:  M0DEL1 


Dependent  Variable:  noque 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

5 

17.47113 

3.49423 

85.89 

<.0001 

Error 

13 

0.52887 

0.04068 

Corrected  Total 

18 

18.00000 

Root  MSE 

0.20170 

R- Square 

0.9706 

Dependent 

Mean 

-2.4688E-16 

Adj  R-Sq 

0.9593 

Coeff  Var 

-8.16996E16 

Parameter  Estimates 


Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  |t| 

Intercept 

1 

-2.4688E-16 

0,04627 

-0.00 

1 .0000 

car 

1 

0.96826 

0.07131 

13.58 

<.0001 

drd 

1 

0.14770 

0.07131 

2.07 

0.0588 

eei 

1 

0.50875 

0.07131 

7.13 

<,0001 

usp 

1 

0.97401 

0.07131 

13,66 

<.0001 

mrs 

1 

-0.13047 

0.07131 

-1.83 

0.0903 

CoHOST  Multiple  Linear  Regression  Model/  Standardized  Data-  Variable  TSKINT 


The  REG  Procedure 


11:07  Thursday,  July  5, 


9 

2001 


Correlation 


Variable 

car 

drd 

eei 

usp 

mrs 

tskint 

car 

1.0000 

0.0000 

0.0000 

0.0000 

0.0000 

-0.1105 

drd 

0.0000 

1.0000 

0.0000 

0.0000 

0.0000 

-0.2385 

eei 

0.0000 

0.0000 

1.0000 

0.0000 

0.0000 

-0.3548 

usp 

0.0000 

0.0000 

0.0000 

1.0000 

0.0000 

-0,7038 

mrs 

0.0000 

0.0000 

0.0000 

0.0000 

1.0000 

0.1105 

tskint 

-0.1105 

-0.2385 

-0.3548 

-0.7038 

0.1105 

1 .0000 

CoHOST  Multiple 

Linear  Regression  Model/  Standardized  Data- 

Variable  TSKINT 

10 

The  REG  Procedure 


11:07  Thursday,  July  5,  2001 


Model:  M0DEL1 


Dependent  Variable:  tskint 
Analysis  of  Variance 


Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Pr  >  F 

Model 

5 

12.64515 

2.52903 

6.14 

0.0039 

Error 

13 

5.35485 

0.41191 

Corrected 

Total 

18 

18.00000 

Root  MSE 

0.64180 

R-Square 

0.7025 

Dependent 

Mean 

3.21512E-14 

Ad]  R-Sq 

0.5881 

Coeff  Var 

1.996204E15 

Parameter  Estimates 


Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  |t| 

Intercept 

1 

3.21512E-14 

0.14724 

0.00 

1.0000 

car 

1 

-0.16577 

0.22691 

-0.73 

0.4780 

drd 

1 

-0.35771 

0.22691 

-1.58 

0.1389 

eei 

1 

-0.53221 

0.22691 

-2.35 

0.0355 

usp 

1 

-1.05569 

0.22691 

-4.65 

0.0005 

mrs 

1 

0.16577 

0.22691 

0,73 

0,4780 

CoHOST  Multiple  Linear  Regression  Model/  Standardized  Data- 

Variable  TSKSUP 

Variable 

car 

drd 

The  REG  Procedure 

Correlation 

eei 

usp 

11:07  Thursday,  July 

mrs 

car 

1.0000 

0.0000 

0.0000 

0.0000 

0.0000 

drd 

0.0000 

1.0000 

0.0000 

0.0000 

0.0000 

eei 

0.0000 

0.0000 

1.0000 

0.0000 

0.0000 

usp 

0.0000 

0.0000 

0.0000 

1.0000 

0.0000 

nrs 

0.0000 

0.0000 

0.0000 

0.0000 

1.0000 

tsksup 

-0.1513 

-0.4282 

-0.1205 

-0.2539 

-0.2231 

CoHOST  Multiple  Linear  Regression  Model/  Standardized  Data>  Variable  TSKSUP 

11:07  Thursday,  July 

The  REG  Procedure 
Model:  M0DEL1 


Dependent  Variable:  tsksup 


Analysis  of  Variance 

Sum  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value 

Model 

5 

6.03026 

1 .20605 

1.31 

Error 

13 

11.96974 

0.92075 

Corrected 

Total 

18 

18,00000 

Root  MSE 

0.95956 

R- Square 

0.3350 

Dependent 

Mean 

2.03638E-15 

Adj  R-Sq 

0.0793 

Coeff  Var 

4.712065E16 

Parameter  Estimates 

Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  lt| 

Intercept 

1 

2.03638E-15 

0.22014 

0.00 

1 .0000 

car 

1 

-0.22694 

0.33925 

-0.67 

0.5152 

drd 

1 

-0.64235 

0.33925 

-1 .89 

0.0808 

eei 

1 

-0.18078 

0.33925 

-0.53 

0.6031 

usp 

1 

-0.38079 

0.33925 

-1.12 

0.2820 

mrs 

1 

-0.33464 

0.33925 

-0.99 

0.3419 

11 

5,  2001 


tsksup 

-0.1513 

-0.4282 

-0.1205 

-0.2539 

-0.2231 

1.0000 


12 

5,  2001 
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CoHOST  Multiple  Linear  Regression  Model/  Standardized  Data-  Variable  TSKDRP  13 

11:07  Thursday,  July  5,  2001 

The  REG  Procedure 
Correlation 


Variable 

car 

drd 

eei 

usp 

mrs 

tskdrp 

car 

1.0000 

0.0000 

0.0000 

0.0000 

0.0000 

0.4741 

drd 

0.0000 

1.0000 

0.0000 

0.0000 

0.0000 

0.2275 

eei 

0.0000 

0.0000 

1 .0000 

0.0000 

0.0000 

0.2275 

usp 

0.0000 

0.0000 

0.0000 

1.0000 

0.0000 

0,7812 

mrs 

0.0000 

0.0000 

0.0000 

0.0000 

1.0000 

0.0482 

tskdrp 

0.4741 

0.2275 

0.2275 

0.7812 

0.0482 

1.0000 

Cohost  Multiple  Linear  Regression  Model/  Standardized  Data 

-  Variable  TSKDRP 

14 

11:07  Thursday,  July  5,  2001 

The  REG  Procedure 

Model;  M0DEL1 

Dependent  Variable: 

tskdrp 

Analysis  of  Variance 

Sura  of 

Mean 

Source 

DF 

Squares 

Square 

F  Value  Pr  >  F 

Model 

5 

16.93701 

3.38740 

41.43  <.0001 

Error 

13 

1 .06299 

0.08177 

Corrected  Total 

18 

18.00000 

Root 

MSE 

0.28595 

R- Square 

0.9409 

Dependent  Mean 
Coeff  Var 


-8.6188E-16 
-3.31775E16 
Parameter  Estimates 


Adj  R-Sq  0.9182 


Parameter 

Standard 

Variable 

DF 

Estimate 

Error 

t  Value 

Pr  >  lt| 

Intercept 

1 

-8.6188E-16 

0.06560 

-0.00 

1.0000 

car 

1 

0.71117 

0.10110 

7,03 

<.0001 

drd 

1 

0.34129 

0.10110 

3.38 

0.0050 

eei 

1 

0.34129 

0.10110 

3.38 

0.0050 

usp 

1 

1.17183 

0.10110 

11.59 

<,0001 

mrs 

1 

0.07229 

0.10110 

0.72 

0.4872 

A 
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SAS  Results  -  P  Values  For  Variable  MRS. 


Variable 

DF 

Parameter  Estimates 
Parameter  Standard 

Estimate  Error 

t  Value 

Pr  >  |t| 

mrs 

1 

Dependent  Variable:  tltt 
0.46996  0.25776 

1.82 

0.0913 

nrs 

1 

Dependent  Variable:  tlksa 
-0.08609  0.04221 

-2.04 

0.0623 

nrs 

1 

Dependent  Variable:  util 
-0.41098  0.15650 

-2.63 

0.0209* 

nrs 

1 

Dependent  Variable:  noque 
-0.13047  0.07131 

-1.83 

0.0903 

nrs 

1 

Dependent  Variable:  tskint 
0.16577  0.22691 

0.73 

0.4780 

mrs 

1 

Dependent  Variable:  tsksup 
-0.33464  0.33925 

-0.99 

0.3419 

mrs 

1 

Dependent  Variable:  tskdrp 
0.07229  0.10110 

0.72 

0.4872 

*  The  smallest  P  value  is  .0209  for  the  dependent  variable  UTIL.  This  value  is  therefore  used  as 
the  threshold  for  significance  to  determine  which  of  the  independent  variables  will  be  used  in  the 
regression  equation  for  each  dependent  variable. 


* 


A 
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Appendix  G  Determination  of  Standardized  Response  Surface  Maximum  Values  For  Each  DV. 


% 


A 
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Dependent  Variable  Taskload  From  Tasktime  -  TLTT. 
The  standardized  regression  equation  for  TLTT  is: 
TLTT 


-0.91927USP  -8.06370E-15 

Taking  the  partial  derivative  of  each  independent  variable  gives: 

TLTTW  =  -0.91927 

Since  there  is  only  one  equation  and  one  unknown  the  result  is  determined  by  inspection: 
USP  =  -0.91927 


k 
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Dependent  Variable  Taskload  From  KSA  -  TLKSA. 

The  standardized  regression  equation  for  TLKSA  is: 

TLKSA  =  +1.48617USP  -1.19310E-15 

^  By  inspection  USP  =  1 .48617 


k 
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Dependent  Variable  Utilization  -  UTE.. 

The  standardized  regression  equation  for  UTIL  is: 

UTIL  =  -0.7I130CAR-0.64808DRD  -0.90099USP  -1.33020E-I3 

Taking  the  partial  derivative  of  each  independent  variable  gives: 

UTIL’car  =-0.71130  -0.64808DRD  -0.90099USP 

UTIL’drd  =-0.71130CAR  -0.64808  -0.90099USP 

UTIL’usp  =-0.71130CAR  -0.64808DRD  -0.90099 

Setting  each  partial  derivative  equal  to  zero  gives: 

-0.71130  -0.64808DRD  -0.90099USP  =  0 

-0.71130CAR  -0.64808  -0.90099USP  =  0 

-0.71130CAR  -0.64808DRD  -0.90099  =  0 

Expressing  these  equations  in  matrix  form  and  solving  (MathSoft,  2001)  for  the  3  unknowns  gives: 


Taskload  From  Utilization  -  UTIL 


0 

-0.64808  -0.90099) 

r0.71130A 

M:= 

-0.71130 

0 

-0.90099 

V  := 

0.64808 

1-0.71130  -0.64808 

0  J 

.0.90099; 

soln  ;=lsolve(M,v) 

^-0.589> 

soln  =  -0.744 
^-0.254; 

Determination  of  Maximum  Regression  Values  For  UTIL 

Expressing  the  results  back  into  equation  form  gives  the  maximum  value  in  the  response  surface  for  each 
independent  measure  for  the  dependent  measure  UTIL: 

CAR  =  -0.589 
DRD  =  -0.744 
USP  =  -0.254 
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Dependent  Variable  Number  of  Queues  -  NOQUE. 

The  standardized  regression  equation  for  NOQUE  from  Equations  6  is: 

NOQUE  =  0.96826CAR  +0.50875EEI  +0.97401USP-2.46880E-16 

Taking  the  partial  derivative  of  each  independent  variable  gives: 

NOQUE’car  =0.96826  +  0.50875EEI  +  0.97401USP 

NOQUE’eei  =0.96826CAR  +  0.50875  +  0.97401USP 

NOQUE’usp  =0.96826CAR  +  0.50875EEI  +  0.97401 

Setting  each  partial  derivative  equal  to  zero  gives: 

0.96826  +  0.50875EEI  +  0.97401USP  =  0 

0.96826CAR  +  0.50875  +  0.97401USP  =  0 

0.96826CAR  +  0.50875EEI  +  0.97401  =  0 

Expressing  these  equations  in  matrix  form  and  solving  (MathSoft,  2001)  for  the  4  unknowns  gives: 


Taskload  From  Number  of  Queues  -  NOQUE 


^  0  0.50875  0.97401> 

^-0.50875> 

M:= 

0.96826  0  0.97401 

V  := 

-0.50875 

^0.96826  0.50875  0  ; 

^-0.97401; 

soln  :=lsolve(M,v) 


soln  = 


^-0.503> 

-0.957 

^-0.022; 


Determination  of  Maximum  Regression  Values  For  NOQUE 


^  Expressing  the  results  back  into  equation  form  gives  the  maximum  value  in  the  response  surface  for  each 

independent  measure  for  the  dependent  measure  NOQUE: 

CAR  =  -0.503 

«  EEI  =  -0.957 

USP  =  -0.022 
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Dependent  Variable  Number  of  Task  Interrupts  -  TSKINT. 

The  standardized  regression  equation  for  TSKINT  is: 

TSKINT  =  +1.05569USP  +3.21512E-14 

By  Inspection, 

USP  =  1.05569 


k 
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Dependent  Variable  Nxunber  of  Task  Suspensions  -  TSKSUP. 

The  standardized  regression  equation  for  TSKSUP  from  Equations  6  is: 
TSKSUP  = 


+  2.03638E-15 


This  variable  does  not  significantly  respond  to  any  of  the  IV’s. 


Dependent  Variable  Number  of  Task  Drops  -  TSKDRP. 

The  standardized  regression  equation  for  TSKDRP  from  Equations  6  is: 

TSKDRP  =0.71117CAR+0.34129DRD+0.34129EEI  +1.17183USP  -8.61880E-16 


4 


Taking  the  partial  derivative  of  each  independent  variable  gives: 


TSKDRP’car  =  0.71 1 17  +0.34129DRD 

TSKDRP’drd  =  0.71 1 17CAR  +0.34129 
TSKDRP’eei  =0.71117CAR  +0.34129DRD 
TSKDRP’usp  =0.71117CAR  +0.34129DRD 


+0.34129EEI 

+0.34129EEI 

+0.34129 

+0.34129EEI 


+1.17183USP 

+1.17183USP 

+1.17183USP 

+1.17183 


Setting  each  partial  derivative  equal  to  zero  gives: 


0.71117 

0.71117CAR 

0.71117CAR 

0.71117CAR 


+0.34129DRD  +0.34129EEI 
+0.34129  +0.34129EEI 

+0.34129DRD  +0.34129 
+0.34129DRD  +0.34129EEI 


+1.17183USP  =0 
+1.17183USP  =0 
+1.17183USP  =0 
+1.17183  =0 


Expressing  these  equations  in  matrix  form  and  solving  (MathSoft,  2001)  for  the  4  unknowns  gives: 


Taskload  From  Number  of  Tasks  Dropped  -  TSKDRP 


^  0  0.34129  0.34129  1.17183^ 

^-0.71117'^ 

0.71117  0  0.34129  1.17183 

-0.34129 

M:= 

0.71117  0.34129  0  1.17183 

V  := 

-0.34129 

^0.71117  0.34129  0.34129  0  > 

^-1.17183> 

4 


soln  :=lsolve(M,v) 


(  -0.203"\ 


soln  = 


-1.506 

-1.506 


0.27  ) 


Determination  of  Maximum  Regression  Values  For  TSKDRP 


Expressing  the  results  back  into  equation  form  gives  the  maximum  value  in  the  response  surface  for  each 
independent  measvue  for  the  dependent  measure  TSKDRP: 

CAR  =  -0.203 
DRD  =  -1.506 
EEI  =  -1.506 
USP  =  0.27 
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Appendix  H  -  Determination  of  Significance  of  Main  Effects  for  Each  Dependent  Variable. 


% 


IL 


* 
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SAS  GLM  for  ANOVA  Run  -  Original  Data. 

options  pageno=l  f ormdlim= ' -  * ; 

TITLE  'CoHOST  1/4  Replicate  of  2**5  Design'; 

DA.TA  cohost; 

INPUT  car  drd  eei  usp  mrs  subj  tltt  tlksa  util  noque  tskint  tsksup  tskdrp; 

*  Independent  variables  -  Factors:; 

*  Factor  A;  CAR  -  Communicate  and  Report; 

*  Factor  B:  DRD  -  Decide  and  Recommend  /  Direct; 

*  Factor  C:  EEI  -  Evaluate  and  Estimate  Impact; 

*  Factor  D:  USP  -  Identify  /  Understand  Situational  Picture; 

*  Factor  E:  MRS  -  Manage  Resources; 

*  Simulated  subjects:  subj; 


*  Dependent  variables:; 

*  TLTT  -  Taskload  from  VJorkload  in  the  Task  Time  runs; 

*  TLKSA  -  Taskload  from  Workload  in  the  KSA  /  JASS  data  runs; 

*  UTIL  -  %  Utilization  over  run; 

*  NOQUE  -  Number  of  queues  generated  during  the  run,  ; 

*  i.e.,  the  number  of  times  a  task  v/as  queued  up; 

*  TSKINT  "  Number  of  times  a  task  was  interrupted; 

*  TSKSUP  -  Number  of  times  a  task  was  suspended; 

*  TSKDRP  -  Number  of  times  a  task  was  dropped; 

Provide  input  data  as  CARDS  statements  ; 

*  1/4  replicate  portion;  ; 

*  5  Factors  -  2  levels /factor=  32  Treatment  Combinations  x  ; 

*  1  subject  per  cell  =  ; 

*  32  Treatments  if  full  factorial; 

*  8  Treatments  for  1/4  fractional  factorial; 

*  Central  composite  center  point  treatment:  ; 

*  11  treatments  on  center  point  to  provide  3rd  treatment  level; 

* . 

CARDS; 


1 

-1 

-1 

-1 

-1 

1 

39,275 

31.884 

0.9250 

49.33 

90.33 

10.47 

9.60 

-1 

1 

1 

-1 

-1 

1 

40.084 

32.625 

0.9256 

47.13 

89.07 

10.33 

9.40 

1 

1 

-1 

1 

-X 

1 

38.588 

46.259 

0.9213 

55.13 

88.47 

10.47 

14.20 

-1 

-1 

1 

1 

-1 

1 

36.793 

46.988 

0.9242 

51.73 

88.47 

10.67 

12.20 

-1 

1 

-1 

-1 

1 

1 

39.810 

31.557 

0.9242 

42.80 

90.87 

10.40 

8.93 

1 

-1 

1 

-1 

1 

1 

39.439 

32.952 

0.9244 

50.60 

90.27 

10.80 

10.93 

-1 

“1 

-1 

1 

1 

1 

39.217 

45.920 

0.9240 

48.27 

89.07 

10.40 

11.07 

1 

1 

1 

1 

1 

1 

38.891 

44.180 

0.9183 

58.47 

87.27 

9.47 

15.33 

0 

0 

0 

0 

0 

1 

39.439 

39.439 

0.9244 

50.60 

90.27 

10.80 

10.93 

0 

0 

0 

0 

0 

2 

39.391 

39.391 

0.9240 

49.87 

90.47 

10.60 

11.13 

0 

0 

0 

0 

0 

3 

39.372 

39.372 

0.9242 

49.87 

89.87 

10.80 

11.13 

0 

0 

0 

0 

0 

4 

39.351 

39.351 

0.9243 

49.53 

90.13 

10.93 

10.53 

0 

0 

0 

0 

0 

5 

39.407 

39.407 

0.9246 

50.93 

89.60 

10.73 

10.47 

0 

0 

0 

0 

0 

6 

39.367 

39.367 

0.9244 

49.67 

89.60 

10,80 

10.93 

0 

0 

0 

0 

0 

7 

39.320 

39.320 

0.9246 

49.07 

90.27 

10.73 

10.93 

0 

0 

0 

0 

0 

8 

39.346 

39.346 

0.9239 

49.73 

89.87 

10.87 

10.60 

0 

0 

0 

0 

0 

9 

40.207 

40.207 

0.9235 

50.00 

89.87 

10,80 

10.87 

0 

0 

0 

0 

0 

10 

39.340 

39.340 

0.9238 

50.07 

89.67 

10.73 

11.20 

0 

0 

0 

0 

0 

11 

39.382 

39.382 

0.9246 

50.40 

89.80 

10.60 

10.80 

proc  glm  data=cohost; 

TITLE  'Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Taskload  from  Task  Time' 
class  car  drd  eei  usp  mrs; 

model  tltt  =  car  drd  eei  usp  mrs  drd*eei  drd*nirs; 


proc  glm  data=cohost; 

TITLE  'Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Taskload  from  KSA  Values' 
class  car  drd  eei  usp  mrs; 

model  tlksa  =  car  drd  eei  usp  mrs  drd*eei  drd*mrs; 

run; 

proc  glm  data=cohost; 

TITLE  'Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Utilization'; 
class  car  drd  eei  usp  mrs; 

model  util  =  car  drd  eei  usp  mrs  drd*eei  drd*mrs; 

run; 
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proc  glm  data=cohost; 

TITLE  ’Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Number  of  Queues’ 
class  car  drd  eei  usp  mrs; 

model  noc(ue  =  car  drd  eei  usp  mrs  drd*eei  drd*inrs; 

run; 

proc  gla  data-cohost; 

TITLE  'Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Task  Interrupts'; 
class  car  drd  eei  usp  mrs; 

model  tskint  =  car  drd  eei  usp  mrs  drd*eei  drd*mrs; 

run; 

proc  glm  data=cohost; 

TITLE  'Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Task  Suspensions' 
class  car  drd  eei  usp  mrs; 

model  tsksup  =  car  drd  eei  usp  mrs  drd*eei  drd*mrs; 


proc  glm  data=cohost; 

TITLE  'Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Task  Drops'; 
class  car  drd  eei  usp  mrs; 

model  tskdrp  =  car  drd  eei  usp  mrs  drd*  eei  drd*inrs; 


SAS  GLM  for  ANOVA  Run  Output  -  Original  Data. 


Augaented  1/4  Replicate  of  2**5  Design  on  COHOST  Siaulation  -  Taskload  from  Task  Time  1 

08:54  Thursday,  June  21,  2001 

The  GLM  Procedure 


Class  Level  Information 


Class 

Levels 

Values 

car 

3 

-1  0  1 

drd 

3 

-1  0  1 

eei 

3 

-1  0  1 

usp 

3 

-1  0  1 

mrs 

3 

-1  0  1 

Number  of  observations  19 


% 


Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation 

The  GLM  Procedure 


Taskload  from  Task  Time 
08:54  Thursday,  June  21, 


Dependent  Variable:  tltt 


Source 

DF 

Sum  of 
Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

8 

8.07562780 

1.00945347 

15.63 

0.0001 

Error 

Corrected  Total 

Source 

R-Square 

0.925969 

10  0.64564273  0.06456427 

18  8.72127053 

Coeff  Van  Root  MSE  tltt  Mean 

0.647142  0.254095  39.26416 

DF  Type  I  SS  Mean  Square  F  Value 

Pr  >  F 

car 

2 

0.88817905 

0.44408952 

6.88 

0.0132 

drd 

1 

0.87715013 

0.87715013 

13.59 

0.0042 

eei 

1 

0.35406113 

0.35406113 

5.48 

0.0412 

usp 

1 

3.27552013 

3.27552013 

50.73 

<.0001 

mrs 

1 

0.85608612 

0.85608612 

13.26 

0.0045 

drd*eei 

1 

1.00607112 

1,00607112 

15.58 

0.0027 

drd*mrs 

1 

0.81856013 

0.81856013 

12.68 

0.0052 

Source 

DF 

Type  III  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

0.01044012 

0.01044012 

0.16 

0.6961 

drd 

1 

0.87715013 

0.87715013 

13.59 

0.0042 

eei 

1 

0.35406113 

0.35406113 

5.48 

0.0412 

usp 

1 

3.27552013 

3.27552013 

50.73 

<.0001 

mrs 

1 

0.85608612 

0.85608612 

13.26 

0.0045 

drd*eei 

1 

1.00607112 

1.00607112 

15.58 

0.0027 

drd*mrs 

1 

0.81856013 

0.81856013 

12.68 

0.0052 

2 

2001 
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Augmented  1/4  Replicate  of 


Augmented  1/4  Replicate  of 


Dependent  Variable:  tlksa 

Source 

Model 

Error 

Corrected  Total 

R- Square 
0.998282 

Source 

car 

drd 

eei 

usp 

mrs 

drd*eei 

drd*mr8 

Source 

car 

drd 

eei 

usp 

mrs 

drd*eei 

drd^mrs 


2**5  Design  on  COHOST  Simulation  -  Taskload  from  KSA  Values  3 

08:54  Thursday,  June  21,  2001 

The  GLM  Procedure 
Class  Level  Information 


Class 

Levels 

Values 

car 

3 

-1  0  1 

drd 

3 

-1  0  1 

eei 

3 

-1  0  1 

usp 

3 

-1  0  1 

mrs 

3 

-1  0  1 

Number  of  observations  19 


2**5  Design  on  COHOST  Simulation  ■  Taskload  from  KSA  Values 

08:54  Thursday,  June  21, 


The  GLM  Procedure 


4 

2001 


DF 

Sum  of 
Squares 

Mean  Square 

F  Value 

Pr  >  F 

8 

375.2058530 

46.9007316 

726.42 

<.0001 

10  0.6456427  0.0645643 

18  375.8514957 

Coeff  Var  Root  MSE  tlksa  Mean 

0.646910  0.254095  39.27826 

DF  Type  I  SS  Mean  Square  F  Value 

Pr  >  F 

2 

1.1596252 

0.5798126 

8.98 

0.0059 

1 

1.2191411 

1.2191411 

18.88 

0.0015 

1 

0.1582031 

0.1582031 

2.45 

0,1486 

1 

368.9550301 

368.9550301 

5714.54 

<.0001 

1 

1.2379511 

1.2379511 

19.17 

0.0014 

1 

1.2379511 

1.2379511 

19,17 

0.0014 

1 

1.2379511 

1.2379511 

19.17 

0.0014 

DF 

Type  III  SS 

Mean  Square 

F  Value 

Pr  >  F 

1 

0.4117781 

0.4117781 

6.38 

0.0301 

1 

1.2191411 

1.2191411 

18.88 

0.0015 

1 

0.1582031 

0.1582031 

2.45 

0.1486 

1 

368.9550301 

368.9550301 

5714.54 

<.0001 

1 

1.2379511 

1.2379511 

19.17 

0.0014 

1 

1.2379511 

1.2379511 

19.17 

0.0014 

1 

1.2379511 

1.2379511 

19.17 

0.0014 

s 


p 


% 


Augioented  1/4  Replicate  of  2**5  Design  on  COHOST  Sinulation  -  Utilization 

08:54  Thursday,  June  21 

The  GLM  Procedure 


Class  Level  Information 


Class 

Levels 

Values 

car 

3 

-1  0  1 

drd 

3 

-1  0  1 

eei 

3 

-1  0  1 

usp 

3 

-1  0  1 

mrs 

3 

-1  0  1 

Number  of  observations  19 


Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Utilization 


Dependent  Variable:  util 


The  GLM  Procedure 


08:54  Thursday,  June  21 


Source 

DF 

Sum  of 
Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

8 

0.00004368 

0.00000546 

40.47 

<.0001 

Error 

Corrected  Total 

Source 

R- Square 
0.970038 

10  0.00000135  0.00000013 

18  0.00004503 

Coeff  Var  Root  MSE  util  Mean 

0.039757  0.000367  0.923858 

OF  Type  I  SS  Mean  Square  F  Value 

Pr  >  F 

car 

2 

0.00001335 

0.00000667 

49.47 

<.0001 

drd 

1 

0.00000841 

0.00000641 

62.30 

<.0001 

eei 

1 

0.00000050 

0.00000050 

3.71 

0.0831 

usp 

1 

0.00001624 

0.00001624 

120.41 

<.0001 

mrs 

1 

0.00000338 

0.00000338 

25.05 

0.0005 

drd*eei 

1 

0.00000018 

0.00000018 

1.33 

0.2749 

drd*mrs 

1 

0.00000162 

0.00000162 

12.01 

0.0061 

Source 

DF 

Type  III  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

0.00001013 

0.00001013 

75.05 

<.0001 

drd 

1 

0.00000841 

0.00000841 

62.30 

<.0001 

eei 

1 

0.00000050 

0.00000050 

3.71 

0.0831 

usp 

1 

0.00001624 

0.00001624 

120.41 

<.0001 

mrs 

1 

0.00000336 

0.00000338 

25.05 

0.0005 

drd*eei 

1 

0.00000018 

0.00000018 

1.33 

0.2749 

drd*mrs 

1 

0.00000162 

0.00000162 

12.01 

0.0061 

5 

,  2001 


6 

,  2001 
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Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Number  of  Queues  7 

08:54  Thursday,  June  21,  2001 

The  GLM  Procedure 


Class  Level  Information 


Class 

Levels 

Values 

car 

3 

-1  0  1 

drd 

3 

-1  0  1 

eel 

3 

-1  0  1 

usp 

3 

-1  0  1 

mrs 

3 

-1  0  1 

Number  of 

observations  19 

Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Number  of  Queues 

08:54  Thursday,  June  21, 


The  GLM  Procedure 


Dependent  Variable:  noque 


Source 

DF 

Sum  of 
Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

8 

164.3965981 

20.5495748 

76.53 

<.0001 

Error 

Corrected  Total 

Source 

R- Square 
0.983930 

10  2.6850545  0.2685055 

18  167.0816526 

Coeff  Var  Root  MSE  noque  Mean 

1.032871  0.518175  50.16842 

DF  Type  I  SS  Mean  Square  F  Value 

Pr  >  F 

car 

2 

70.58364809 

35.29182404 

131.44 

<.0001 

drd 

1 

1 .62000000 

1.62000000 

6.03 

0.0339 

eei 

1 

19.22000000 

19.22000000 

71.58 

<.0001 

usp 

1 

70.44845000 

70.44845000 

262.37 

<.0001 

mrs 

1 

1 .26405000 

1.26405000 

4.71 

0.0552 

drd*eei 

1 

1 .08045000 

1.08045000 

4.02 

0.0727 

drd*mrs 

1 

0.18000000 

0.18000000 

0.67 

0.4320 

Source 

DF 

Type  III  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

69.62000000 

69.62000000 

259.29 

<.0001 

drd 

1 

1 .62000000 

1 .62000000 

6.03 

0.0339 

eei 

1 

19.22000000 

19.22000000 

71.58 

<.0001 

usp 

1 

70.44845000 

70.44845000 

262.37 

<.0001 

mrs 

1 

1 .26405000 

1.26405000 

4.71 

0.0552 

drd*eei 

1 

1 .08045000 

1.08045000 

4.02 

0.0727 

drd*rors 

1 

0.18000000 

0.18000000 

0.67 

0.4320 

8 

2001 


171 


<$ 
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Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Task  Interrupts  9 

08:54  Thursday,  June  21,  2001 

The  GLM  Procedure 
Class  Level  Information 


Class 

Levels 

Values 

car 

3 

-1  0  1 

drd 

3 

-1  0  1 

eei 

3 

-1  0  1 

usp 

3 

-1  0  1 

mrs 

3 

-1  0  1 

Number  of  observations  19 


Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Task  Interrupts 

08:54  Thursday,  June  21, 


The  GLM  Procedure 


Dependent  Variable:  tskint 


Source 

DF 

Sum  of 
Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

8 

12.42864498 

1.55358062 

17.80 

<.0001 

Error 

Corrected 

Source 

Total 

R -Square 
0.934396 

10  0.87261818  0.08726182 

18  13.30126316 

Coeff  Var  Root  MSE  tskint  Mean 

0.329526  0.295401  89.64421 

DF  Type  I  SS  Mean  Square  F  Value 

Pr  >  F 

car 

2 

2.56194498 

1.28097249 

14.68 

0.0011 

drd 

1 

0.75645000 

0.75645000 

8.67 

0.0147 

eei 

1 

1 .67445000 

1.67445000 

19.19 

0.0014 

usp 

1 

6.58845000 

6.58845000 

75.50 

<.0001 

mrs 

1 

0.16245000 

0.16245000 

1.86 

0.2024 

drd*eei 

1 

0.68445000 

0.68445000 

7.84 

0.0188 

drd*rars 

1 

0.00045000 

0.00045000 

0.01 

0.9442 

Source 

DF 

Type  III  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

0.16245000 

0.16245000 

1.86 

0.2024 

drd 

1 

0.75645000 

0.75645000 

8.67 

0.0147 

eei 

1 

1 ,67445000 

1.67445000 

19.19 

0.0014 

usp 

1 

6.58845000 

6.58845000 

75.50 

<.0001 

mrs 

1 

0.16245000 

0.16245000 

1.86 

0.2024 

drd*eei 

1 

0.68445000 

0.68445000 

7.84 

0.0188 

drd*mrs 

1 

0.00045000 

0.00045000 

0.01 

0.9442 

10 

2001 
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Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  •  Task  Suspensions  11 

08:54  Thursday,  June  21,  2001 

The  GLM  Procedure 
Class  Level  Information 


Class 

Levels 

Values 

car 

3 

-1  0  1 

drd 

3 

-1  0  1 

eei 

3 

-1  0  1 

usp 

3 

-1  0  1 

mrs 

3 

-1  0  1 

Number  of  observations  19 


Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Task  Suspensions 

08:54  Thursday,  June  21, 


The  GLM  Procedure 


Dependent  Variable:  tsksup 


Source 

DF 

Sum  of 
Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

8 

1.79978182 

0.22497273 

22.23 

<.0001 

Error 

Corrected 

Source 

Total 

R>Square 

0.946755 

10  0.10121818  0.01012182 

18  1.90100000 

Coeff  Var  Root  MSE  tsksup  Mean 

0.949125  0.100607  10.60000 

DF  Type  I  SS  Mean  Square  F  Value 

Pr  >  F 

car 

2 

0.73530682 

0.36765341 

36.32 

<.0001 

drd 

1 

0.34861250 

0.34861250 

34.44 

0.0002 

eei 

1 

0.02761250 

0.02761250 

2.73 

0.1296 

usp 

1 

0.12251250 

0.12251250 

12.10 

0.0059 

mrs 

1 

0.09461250 

0.09461250 

9.35 

0.0121 

drd*eei 

1 

0.34861250 

0.34861250 

34.44 

0.0002 

drd*mrs 

1 

0.12251250 

0.12251250 

12.10 

0.0059 

Source 

DF 

Type  III  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

0.04351250 

0.04351250 

4.30 

0.0649 

drd 

1 

0.34861250 

0.34861250 

34.44 

0.0002 

eei 

1 

0.02761250 

0.02761250 

2.73 

0.1296 

usp 

1 

0.12251250 

0.12251250 

12.10 

0.0059 

mrs 

1 

0.09461250 

0.09461250 

9.35 

0.0121 

drd*eei 

1 

0.34861250 

0.34861250 

34,44 

0.0002 

drd*mrs 

1 

0.12251250 

0.12251250 

12.10 

0.0059 

12 

2001 


Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Task  Drops  13 

08:54  Thursday,  June  21,  2001 

The  GLM  Procedure 
Class  Level  Information 


Class 

Levels 

Values 

car 

3 

-1  0  1 

drd 

3 

-1  0  1 

eei 

3 

-1  0  1 

usp 

3 

-1  0  1 

mrs 

3 

-1  0  1 

Number  of  observations  19 


Augmented  1/4  Replicate  of 

2**5  Design  on  COHOST  Simulation 

-  Task  Drops 

08:54  Thursday 

,  June  2 

The  GLM  Procedure 

Dependent  Variable:  tskdrp 

Sum  of 

Source 

DF 

Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

8 

39.19260096 

4.89907512 

80.57 

<.0001 

Error 

10 

0.60807273 

0.06080727 

Corrected  Total 

18 

39.80067368 

R -Square 

Coeff  Var  Root 

MSE  tskdrp  Mean 

0.984722 

2. 

218598  0.246591  11,11474 

Source 

DF 

Type  I  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

2 

10.56990096 

5.28495048 

86.91 

<.0001 

drd 

1 

2.06045000 

2.06045000 

33.88 

0.0002 

eei 

1 

2.06045000 

2.06045000 

33.88 

0.0002 

usp 

1 

24.29045000 

24.29045000 

399.47 

<.0001 

mrs 

1 

0.09245000 

0.09245000 

1.52 

0.2458 

drd*eei 

1 

0.09245000 

0.09245000 

1.52 

0.2458 

drd*mrs 

1 

0.02645000 

0.02645000 

0.43 

0.5245 

Source 

DF 

Type  III  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

8.94645000 

8.94645000 

147.13 

<.0001 

drd 

1 

2.06045000 

2.06045000 

33.88 

0.0002 

eei 

1 

2.06045000 

2.06045000 

33.88 

0.0002 

usp 

1 

24.29045000 

24.29045000 

399.47 

<.0001 

mrs 

1 

0.09245000 

0.09245000 

1.52 

0.2458 

drd* eei 

1 

0.09245000 

0.09245000 

1.52 

0.2458 

drd*mrs 

1 

0.02645000 

0.02645000 

0.43 

0.5245 

SAS  GLM  for  ANOVA  Run  -  Standardized  Data, 


options  pageno=l  formdlim= ' - ‘ ; 

TITLE  'CoHOST  1/4  Replicate  of  2**5  Design'; 

SATA  cohost; 

INPUT  car  drd  eei  usp  mrs  subj  tltt  tlksa  util  noque  tskint  tsksup  tskdrp; 

*  Independent  variables  -  Factors:; 


CAR  ~  Communicate  and  Report; 

DRD  ~  Decide  and  Recommend  /  Direct; 

EEI  -  Evaluate  and  Estimate  Impact; 

USP  -  Identify  /  Understand  Situational  Picture; 
MRS  “  Manage  Resources; 


Factor  A 
Factor  B 
Factor  C 
Factor  D 
Factor  E 
Simulated  subjects:  subj ; 


*  Dependent  variables:; 

*  TLTT  -  Taskload  from  VJorkload  in  the  Task  Time  runs; 

*  TLKSA  “  Taskload  from  Workload  in  the  KSA  /  JASS  data  runs; 

*  UTIL  -  %  Utilization  over  run; 

*  NOQUE  ~  Number  of  queues  generated  during  the  run,  ; 

*  i.e.,  the  number  of  times  a  task  was  queued  up; 

*  TSKINT  -  Number  of  times  a  task  was  interrupted; 

*  TSKSUP  -  Number  of  times  a  task  was  suspended; 

*  TSKDRP  -  Number  of  times  a  task  was  dropped ; 


*  Provide  input  data  as  CARDS  statements  ; 

*  1/4  replicate  portion:  ; 

*  5  Factors  -  2  levels/f actor=  32  Treatment  Combinations  x  ; 

*  1  subject  per  cell  =  ; 

*  32  Treatments  if  full  factorial; 

*  8  Treatments  for  1/4  fractional  factorial; 

*  Central  composite  center  point  treatment: 

*  11  treatments  on  center  point  to  provide  3rd  treatment  level; 


* . 

CARDS; 


1 

-1 

-1 

-1 

-1 

1 

39.275 

31.884 

0.9250 

49.33 

90,33 

10.47 

9.60 

-1 

1 

1 

-1 

-1 

1 

40.084 

32.625 

0.9256 

47.13 

89.07 

10.33 

9.40 

1 

1 

-1 

1 

-1 

1 

38.588 

46.259 

0.9213 

55.13 

88.47 

10.47 

14.20 

-1 

-1 

1 

1 

-1 

1 

36.793 

46.988 

0.9242 

51.73 

88.47 

10.67 

12.20 

-1 

1 

-1 

-1 

1 

1 

39.810 

31.557 

0.9242 

42.80 

90.87 

10.40 

8.93 

1 

-1 

1 

-1 

1 

1 

39.439 

32.952 

0.9244 

50.60 

90.27 

10.80 

10.93 

-1 

-1 

-1 

1 

1 

1 

39.217 

45.920 

0.9240 

48.27 

89.07 

10.40 

11.07 

1 

1 

1 

1 

1 

1 

38.891 

44.180 

0.9183 

58.47 

87.27 

9.47 

15.33 

0 

0 

0 

0 

0 

1 

39.439 

39.439 

0.9244 

50.60 

90.27 

10,80 

10.93 

0 

0 

0 

0 

0 

2 

39.391 

39.391 

0.9240 

49.87 

90.47 

10.60 

11.13 

0 

0 

0 

0 

0 

3 

39.372 

39.372 

0.9242 

49.87 

89.87 

10.80 

11.13 

0 

0 

0 

0 

0 

4 

39.351 

39.351 

0.9243 

49.53 

90.13 

10.93 

10.53 

0 

0 

0 

0 

0 

5 

39.407 

39.407 

0.9246 

50.93 

89.60 

10.73 

10.47 

0 

0 

0 

0 

0 

6 

39.367 

39.367 

0.9244 

49.67 

89.60 

10.80 

10.93 

0 

0 

0 

0 

0 

7 

39.320 

39.320 

0.9246 

49.07 

90.27 

10.73 

10.93 

0 

0 

0 

0 

0 

8 

39.346 

39.346 

0.9239 

49.73 

89.87 

10.87 

10.60 

0 

0 

0 

0 

0 

9 

40.207 

40.207 

0.9235 

50.00 

89.87 

10.80 

10.87 

0 

0 

0 

0 

0 

10 

39.340 

39.340 

0.9238 

50.07 

89.67 

10.73 

11.20 

0 

0 

0 

0 

0 

11 

39.382 

39.382 

0.9246 

50.40 

89.80 

10.60 

10.80 
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*  standardize  the  data  and  rerun; 

options  pageno^l  fonndlim= ' - ' ; 
proc  standard  mean=0  std=l  out=cohosts  data=cohost; 

var  tltt  tlksa  util  noque  tskint  tsksup  tskdrp; 

run; 


*  AMOVA  Runs: 

proc  g'Xm  data=cohosts ; 

TITLE  'Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Taskload  from  Task  Time* 
class  car  drd  eei  usp; 
model  tltt  =  car  drd  eei  usp; 


proc  pljii  data==cohosts ; 

TITLE  'Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Taskload  from  KSA  Values 
class  car  drd  eei  usp; 
model  tlksa  =  car  drd  eei  usp; 


proc  glm  data=cohosts ; 

TITLE  'Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Utilization* ; 
class  car  drd  eei  usp; 
model  util  =  car  drd  eei  usp; 

run; 

proc  glm  data=cohosts; 

TITLE  'Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Number  of  Queues'; 
class  car  drd  eei  usp; 
model  noque  =  car  drd  eei  usp; 

run; 

proc  glm  data=cohosts; 

TITLE  'Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Task  Interrupts'; 
class  car  drd  eei  usp; 
model  tskint  =  car  drd  eei  usp; 


run; 

proc  glm  data=cohosts ; 

TITLE  'Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Task  Suspensions'; 
class  car  drd  eei  usp; 
model  tsksup  =  car  drd  eei  usp; 


proc  glm  data=cohosts ; 

TITLE  'Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Task  Drops’; 
class  car  drd  eei  usp; 
model  tskdrp  =  car  drd  eei  usp; 

run; 
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SAS  GLM  for  ANOVA  Run  Output  -  Standardized  Data. 


Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Taskload  from  Task  Time  1 

11:07  Thursday,  July  5,  2001 

The  GLM  Procedure 


Class  Level  Information 


Class 

Levels 

Values 

car 

3 

-1  0  1 

drd 

3 

-1  0  1 

eei 

3 

-1  0  1 

usp 

3 

-1  0  1 

Number  of  observations  19 


Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Taskload  from  Task  Time 

11:07  Thursday,  July  5, 


The  GLM  Procedure 


Dependent  Variable:  tltt 

Sum  of 

Source 

DF 

Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

5 

11.13466064 

2.22693213 

4.22 

0.0169 

Error 

13 

6.86533936 

0.52810303 

Corrected  Total 

18 

18.00000000 

R' Square 

Coeff  Var  Root 

MSE  tltt 

Mean 

0.618592 

-8, 

.9777E15  0.726707  -0.000000 

Source 

DF 

Type  I  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

2 

1.83313003 

0.91656501 

1,74 

0.2147 

drd 

1 

1.81036722 

1.81036722 

3,43 

0.0869 

eei 

1 

0.73075365 

0.73075365 

1.38 

0.2606 

usp 

1 

6.76040974 

6.76040974 

12.80 

0.0034 

Source 

DF 

Type  III  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

0.02154758 

0.02154758 

0.04 

0.8430 

drd 

1 

1.81036722 

1 .81036722 

3.43 

0.0869 

eei 

1 

0.73075365 

0.73075365 

1.38 

0.2606 

usp 

1 

6.76040974 

6.76040974 

12.80 

0.0034 

2 

2001 


Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Taskload  from  KSA  Values  3 

11:07  Thursday,  July  5,  2001 

The  GLM  Procedure 


Class  Level  Information 


Class 

Levels 

Values 

car 

3 

-1  0  1 

drd 

3 

-1  0  1 

eei 

3 

-1  0  1 

usp 

3 

-1  0  1 

Number  of  observations  19 


Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Taskload  from  KSA  Values 


The  GLM  Procedure 


11:07  Thursday,  July  5, 


4 

2001 


Dependent  Variable:  tlksa 


Source 

DF 

Sum  of 
Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

5 

17.79121826 

3.55824365 

221.56 

<.0001 

Error 

Corrected  Total 

Source 

R* Square 
0.988401 

13  0.20878174  0.01606013 

18  18.00000000 

Coeff  Var  Root  MSE  tlksa  Mean 

-9.2683E15  0.126729  -0.000000 

DF  Type  1  SS  Mean  Square  F  Value 

Pr  >  F 

car 

2 

0.05553591 

0.02776795 

1 .73 

0.2159 

drd 

1 

0.05838620 

0.05838620 

3.64 

0.0789 

eei 

1 

0.00757655 

0.00757655 

0.47 

0.5042 

usp 

1 

17.66971961 

17.66971961 

1100.22 

<.0001 

Source 

DF 

Type  III  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

0.01972057 

0.01972057 

1.23 

0.2879 

drd 

1 

0.05838620 

0.05838620 

3.64 

0.0789 

eei 

1 

0.00757655 

0.00757655 

0.47 

0.5042 

usp 

1 

17.66971961 

17.66971961 

1100.22 

<.0001 

Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Utilization  5 

11:07  Thursday,  July  5,  2001 

The  GLM  Procedure 


Class  Level  Information 


Class 

Levels 

Values 

car 

3 

-1  0  1 

drd 

3 

-1  0  1 

eei 

3 

-1  0  1 

usp 

3 

-1  0  1 

Number  of  observations  19 


Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Utilization 


The  GLM  Procedure 


11:07  Thursday,  July  5, 


Dependent  Variable:  util 


Source 

DF 

Sum  of 
Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

5 

15.38989002 

3.07797800 

15.33 

<.0001 

Error 

Corrected  Total 

Source 

R- Square 
0.854994 

13  2.61010998  0.20077769 

18  18.00000000 

Coeff  Var  Root  MSE  util  Mean 

-3.3694E14  0.448082  >0.000000 

DF  Type  I  SS  Mean  Square  F  Value 

Pr  >  F 

car 

2 

5.33576962 

2.66788481 

13.29 

0.0007 

drd 

1 

3.36003507 

3.36003507 

16.74 

0.0013 

eei 

1 

0.19988311 

0.19988311 

1.00 

0.3366 

usp 

1 

6.49420222 

6.49420222 

32.35 

<.0001 

Source 

DF 

Type  III  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

4.04763296 

4.04763296 

20.16 

0.0006 

drd 

1 

3.36003507 

3.36003507 

16.74 

0,0013 

eei 

1 

0.19988311 

0.19988311 

1.00 

0.3366 

usp 

1 

6.49420222 

6.49420222 

32.35 

<.0001 

6 

2001 


Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Number  of  Queues  7 

11:07  Thursday,  July  5,  2001 

The  GLM  Procedure 
Class  Level  Information 


Class 

Levels 

Values 

car 

3 

-1  0  1 

drd 

3 

-1  0  1 

eei 

3 

-1  0  1 

usp 

3 

-1  0  1 

Number  of 

observations  19 

1 


Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Number  of  Queues 

11:07  Thursday,  July  5, 


The  GLM  Procedure 


Dependent  Variable:  noque 


Sum  of 

Source 

DF 

Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

5 

17.43876554 

3,48775311 

80.79 

<.0001 

Error 

13 

0.56123446 

0.04317188 

Corrected  Total 

18 

18.00000000 

R- Square  Coeff  Var  Root  MSE  noque  Mean 
0.968820  -7.9907E16  0,207778  -0.000000 


Source 

DF 

Type  I  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

2 

7.60410042 

3.80205021 

88.07 

<.0001 

drd 

1 

0.17452545 

0,17452545 

4.04 

0.0656 

eei 

1 

2.07060437 

2.07060437 

47.96 

<.0001 

usp 

1 

7.58953530 

7.58953530 

175.80 

<.0001 

Source 

DF 

Type  III  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

7.50028492 

7.50028492 

173,73 

<.0001 

drd 

1 

0.17452545 

0.17452545 

4.04 

0.0656 

eei 

1 

2.07060437 

2.07060437 

47.96 

<.0001 

usp 

1 

7.58953530 

7.58953530 

175.80 

<.0001 

8 

2001 


4 


180 


Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Task  Interrupts  9 

11:07  Thursday,  July  5,  2001 

The  GLM  Procedure 


Class  Level  Information 


Class 

Levels 

Values 

car 

3 

•1  0  1 

drd 

3 

-1  0  1 

eei 

3 

-1  0  1 

usp 

3 

-1  0  1 

Number  of 

observations 

19 

1 


Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation 

The  GUM  Procedure 


-  Task  Interrupts 
11:07  Thursday,  July  5, 


Dependent  Variable:  tskint 


Source 

DP 

Sum  of 
Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

5 

15.67244457 

3.13448891 

17.51 

<.0001 

Error 

Corrected 

Source 

Total 

R-Square 

0.870691 

13  2 . 32755543  0 . 1 7904273 

18  18.00000000 

Coeff  Var  Root  MSE  tskint  Mean 

1.30593E15  0.423134  3.2401E-14 

DF  Type  I  SS  Mean  Square  F  Value 

Pr  >  F 

car 

2 

3.46696468 

1 .73348234 

9.68 

0.0027 

drd 

1 

1 .02366970 

1 .02366970 

5.72 

0.0326 

eei 

1 

2.26595772 

2.26595772 

12.66 

0.0035 

usp 

1 

8.91585247 

8.91585247 

49.80 

<.0001 

Source 

DF 

Type  III  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

0.21983626 

0.21983626 

1.23 

0.2879 

drd 

1 

1 .02366970 

1 .02366970 

5.72 

0.0326 

eei 

1 

2.26595772 

2.26595772 

12.66 

0.0035 

usp 

1 

8.91585247 

8.91585247 

49.80 

<.0001 

10 

2001 


4 


181 


Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Task  Suspensions  11 

11:07  Thursday,  July  5,  2001 

The  GLM  Procedure 
Class  Level  Information 


Class 

Levels 

Values 

car 

3 

-1  0  1 

drd 

3 

-1  0  1 

eei 

3 

-1  0  1 

usp 

3 

-1  0  1 

Number  of  observations  19 


y 


Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation 

The  QLM  Procedure 


-  Task  Suspensions 
11:07  Thursday,  July  6, 


Dependent  Variable:  tsksup 


Sum  of 

Source 

DF 

Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

5 

11.68479628 

2.33695926 

4.81 

0.0104 

Error 

13 

6.31520372 

0.48578490 

Corrected  Total 

18 

18.00000000 

R-Square  Coeff  Var  Root  MSE  tsksup  Mean 
0.649155  3.37424E16  0.696983  2.0656E*15 


Source 

DF 

Type  I  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

2 

6.96240017 

3.48120009 

7.17 

0.0080 

drd 

1 

3.30090742 

3.30090742 

6.79 

0.0217 

eei 

1 

0.26145450 

0.26145450 

0.54 

0.4762 

usp 

1 

1.16003419 

1.16003419 

2.39 

0.1463 

Source 

DF 

Type  III  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

0.41200684 

0.41200684 

0.85 

0.3739 

drd 

1 

3.30090742 

3.30090742 

6.79 

0.0217 

eei 

1 

0.26145450 

0.26145450 

0.54 

0.4762 

usp 

1 

1.16003419 

1.16003419 

2.39 

0.1463 

12 

2001 


4 


182 


Augmented  1/4  Replicate  of  2**5  Design  on  COHOST  Simulation  -  Task  Drops  13 

11;07  Thursday,  July  5,  2001 

The  GLM  Procedure 
Class  Level  Information 


Class 

Levels 

Values 

car 

3 

-1  0  1 

drd 

3 

-1  0  1 

eei 

3 

-1  0  1 

usp 

3 

-1  0  1 

Number  of  observations  19 


Augmented  1/4  Replicate  of  2**6  Design  on  COHOST  Simulation  -  Task  Drops  14 

11:07  Thursday,  July  5,  2001 


The  GLM  Procedure 


Dependent  Variable:  tskdrp 


Source 

DF 

Sum  of 
Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

5 

17.62941308 

3.52588262 

123.69 

<.0001 

Error 

Corrected 

Source 

Total 

R-Square 

0.979412 

13  0.37058692  0.02850669 

18  18.00000000 

Coeff  Var  Root  MSE  tskdrp  Mean 

-2.2932E16  0.168839  -0.000000 

DF  Type  I  SS  Mean  Square  F  Value 

Pr  >  F 

car 

2 

4.78027630 

2.39013815 

83.84 

<.0001 

drd 

1 

0.93184604 

0.93184604 

32.69 

<.0001 

eei 

1 

0.93184604 

0.93184604 

32.69 

<.0001 

usp 

1 

10.98544471 

10.98544471 

385.36 

<.0001 

Source 

DF 

Type  III  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

4.04606468 

4.04606468 

141.93 

<.0001 

drd 

1 

0.93184604 

0.93184604 

32.69 

<.0001 

eei 

1 

0.93184604 

0.93184604 

32.69 

<.0001 

usp 

1 

10.98544471 

10.98544471 

385.36 

<.0001 

>1 


183 


Appendix  I  -  Detennination  of  Significance  of  LXDF  And  Regression  For  Each  DV. 

As  previously  discussed,  only  the  main  effects  of  each  treatment  factor  are  of  interest  in 
this  experimental  design  for  each  dependent  variable,  however,  whether  or  not  each  main  effect 
has  a  higher  order  response  is  also  of  interest.  In  order  to  determine  this  the  Lack  of  Fit  (LOF) 
for  each  dependent  variable  is  examined  for  significance.  If  the  test  for  significance  is  positive 
then  that  effect  is  predicted  to  have  at  least  a  quadratic  component  which  can  then  be  examined 
with  a  response  surface  analysis.  The  Type  in  sum  of  squares  results  from  the  PROC  GLM  tests 
in  appendix  H  will  be  used  for  this  LOF  significance  test.  The  procedure  consists  of  first 
summing  the  Type  HI  sum  of  squares  for  the  main  effect  fectors  CAR,  DRD,  EEI,  USP,  and 
MRS.  The  LOF  sum  of  squares  (SS)  is  calculated  by  subtracting  the  SSmain  effects  fi’om  the 
SSmodel  to  product  the  SSlof.  The  mean  square  for  the  LOF  (MSlof)  is  then  calculated  by 
dividing  SSlof  by  the  degrees  of  freedom  for  the  LOF  which  is  3  as  shown  in  the  regression 
summary  table  at  Table  7.  The  F  Ratio  is  then  calculated  by  dividing  the  MSlof  by  the  mean 
square  error  term  from  the  GLM  output. 

This  produces  the  observed  value  for  the  F  ratio  (Fobserved-LOF )  which  is  compared  to  a 
tabled  value  for  the  F  ratio  to  determine  significance  of  the  LOF.  The  numerator  for  the  tabled 
value  is  the  3  degrees  of  fi:eedom  for  LOF  from  Table  7,  and  the  denominator  is  the  10  degrees 
of  freedom  for  the  Error  term  also  from  Table  7  and  from  the  GLM  run.  Looking  this  up  in  an 
appropriate  F  table  (Winer  et  al.,  1991)  (p.  968)  produces  a  tabled  value  for  the  F  ratio  of  6.55  at 
the  .01  level  (the  a  =  .01  level  is  used  as  a  result  of  the  significance  threshold  determined  from 
the  MRS  variable).  This  process  is  illustrated  in  Table  23  using  the  dependent  variable  TLTT  as 
an  example.  The  calculations  and  significance  tests  for  all  the  dependent  variables  is  at  appendix 
I.  The  table  below  summarizes  the  LOF  significance  tests  for  the  dependent  variables.  The  test 
results  in  this  table  indicate  that  only  one  dependent  variable,  NOQUE,  failed  the  LOF  test 
indicating  that  it  is  predicted  to  have  only  a  linear  response.  The  other  six  variables,  TLTT, 
TLKSA,  UTIL,  TSKINT,  TSKSUP,  and  TSKDRP  are  all  predicted  to  have  at  least  a  quadratic 
component  in  their  response  to  the  independent  variables. 
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LOF  For  Dependent  Variable  Taskload  from  Task  Time  (TLTT). 


1  LOF  Calculation  for  T askioad  from  T ask  Time:  I 

1 

The  QLM  Procedure 

Dependent 

Variable: 

tltt 

Sum  of 

Source 

Mean  Square 

F  Value 

Pr  >  F 

Model 

8 

8.0756278 

1 .00945347 

15.63 

0.0001 

Error 

0.64564273 

0,06456427 

Corrected  Total 

18 

8.72127053 

Source 

DF 

Type  III  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

0,01044012 

0.01044012 

0.16 

0.6961 

drd 

1 

0,87715013 

0.87716013 

13.59 

0.0042 

eel 

1 

0.35406113 

0.36406113 

5.48 

0.0412 

usp 

1 

3.27552013 

3.27552013 

60.73 

A 

O 

O 

O 

mrs 

1 

0.85608612 

0.85608612 

13.26 

0.0045 

Sum(IV's)  = 

5.37325763 

LOF  =  SS(model)  -  Sum  SS(IV's)  = 

3 

2.70237017 

0.90079006 

13.952 

Decision  Rule: 

Ho:  Oa'  =  0 

H,:  Oa'^0 

a=  .05 

Fobserved-LOF-TLTT  “ 

13.952 

mm 

*  Ftabled-LOF^  F(3.10)  = 

6.55 

*  Winer,  p.  968  @  .01  level  (l-alpha=.99) 

.’.  Reject  Ho  with  the  result  that  LOF  for  TLTT  is.significant. 

Dependent  Variable: 


LOF  For  Dependent  Variable  Taskload  from  KSA  (TLKSA). 


LOF  Calculation  for  Taskload  from  KSA; 


1  The  GLM  Procedure 

Itlksal 


Source 

DF 

Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

8 

375.205853 

46.9007316 

726.42 

<.0001 

Error 

10 

0.6456427 

0.0645643 

Corrected  Total 

18 

376.8514957 

Source 

DF 

Type  HI  SS 

Mean  Square 

F  Value 

Pr  >  F 

1 

0.4117781 

0.4117781 

6.38 

0.0301 

02 

1 

1.2191411 

eel 

1 

0.1582031 

0.1582031 

2.45 

0.1486 

Sum(IV's)  = 


368.9550301  368.9550301  571 4. 54 I <.0001 


1.2379511 


371.982104 


1.2379511  19.17  0.0014 


LOF  =  SS(model)  -  Sum  SS(IV's)  = 

3 

3.2237495 

1.07458317 

16.644 

Ho;  Oa"  =  0 

H,:  Oa',40 

a=  .05 


D.R.:  IrejectHoifiFobservod 


observtd-LOF-TLKSA  ~ _ _ 

*  Ftabled-ljOF=  F(3.I0)  = _ 

*  Winer,  p.  968  @  .01  level  (l-alpha=.99) 


16.644 

6.55 


Reject  Ho  with  the  result  that  LOF  for  TLKSA  iLsignificant. 

LOF  For  Dependent  Variable  Utilization  (UTIL). 


|LOF  Calculation  for  Utilization:  I 

The  GLM  Procedure 

Dependent 

Variable: 

util 

Sum  of 

Source 

DF 

Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

8 

0.00004368 

0.00000546 

40.47 

<.0001 

Error 

10 

0.00000135 

0.00000013 

Corrected  Total 

18 

0.00004503 

Source 

DF 

Type  HI  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

0.00001013 

0.00001013 

75,05 

<.0001 

drd 

1 

0.00000841 

0.00000841 

62.3 

<.0001 

eel 

1 

0.0000005 

0.0000005 

3.71 

0.0831 

usp 

1 

0.00001624 

0.00001624 

120.41 

<.0001 

mrs 

mi 

0.00000338 

0.00000338 

25.05 

0.0005 

Sum(IV's)  = 

0.00003866 

bbhhi 

■■ii 

LOF  =  SS(model)  -  Sum  SS(IV's)  = 

3 

0.00000502 

1.6733E-06 

12.872 

Decision  Rule: 

Ho:  Oa'  =  0 

H,: 

a=  .05 

Fobserved-LOF-UTIL  ” 

12.872 

*  Ftabled-LOF~  F(3_|0)  = 

6.55 

■"  Winer,  p.  968  @  .01  level  (l-alpha=.99) 

Reject  Ho  with  the  result  that  LOF  for  UTIL  is.significant. 

LOF  For  Dependent  Variable  Number  of  Queues  (NOQUE). 


1  LOF  Calculation  for  Number  of  Queues:  I 

The  GLM  Procedure 

Dependent 

Variable : 

noque 

Sum  of 

Source 

DF 

Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

8 

164.3965981 

20.5495748 

76.53 

<.0001 

Error 

10 

2.6850545 

0.2685055 

Corrected  Total 

18 

167.0816526 

Source 

DF 

Type  in  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

69.62 

69.62 

259.29 

<.0001 

1.62 

6.03 

0.0339 

eel 

1 

19.22 

19.22 

71.58 

<.0001 

usp 

1 

70.44845 

70.44845 

262.37 

<.0001 

mrs 

1 

1 .26405 

1.26405 

4.71 

0.0552 

162.1725 

LOF  =  SS(model)  -  Sum  SS(IV's)  = 

3 

2.2240981 

0.74136603 

2.7611 

Decision  Rule: 

Ho:  Oa'  =  0 

H,: 

a=  .05 

iipii  II 

Fobserved-LOF-NOQUE  ~ 

2.7611 

*  Ftabled-LOF=  F(3,10)  = 

6.55 

*  Winer,  p.  968  @  .01  level  (l-alpha=.99) 

Fail  to  Reject  Ho  with  the  result  that  LOF  for  NOQUE  is  NOT  significant. 

LOF  For  Dependent  Variable  Number  of  Task  Interrupts  (TSKINT). 


|LOF  Calculation  for  Number  of  Tasks  Interrupted:  | 

The  GLM  Procedure 

Dependent 

Variable : 

tskint 

Sum  of 

Source 

DF 

Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

8 

12.42864498 

1 .55358062 

17.8 

<.0001 

Error 

10 

0,87261818 

0.08726182 

Corrected  Total 

18 

13.30126316 

Source 

DF 

Type  III  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

0.16245 

0.16245 

1.86 

0.2024 

drd 

1 

0.75645 

0.75645 

8.67 

0.0147 

eel 

1 

1.67445 

1.67445 

19.19 

0.0014 

usp 

1 

6.58845 

6.58845 

75.5 

<.0001 

mrs 

1 

0.16245 

0.16245 

1.86 

0.2024 

Sum(IV's)  = 

9.34425 

LOF  =  SS(model)  -  Sum  SS(IV's)  = 

3 

3.08439498 

1.02813166 

11.782 

Decision  Rule: 

o 

II 

"to- 

’o 

a 

H,:  o/^O 

O 

II 

e 

D.R. .  I  reject  Hq  if  1  Fobserved  ^  Ftabled 

Fobserved-LOF-TSKINT  = 

11.782 

*  Ftabled-LOF"^  F(3^10)  = 

6.55 

*  Winer,  p.  968  @  .01  level  (l-alpha=.99) 

A  Reject  Ho  with  the  result  that  LOF  for  TSKINT  is.significant. 
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LOF  For  Dependent  Variable  Number  of  Task  Suspensions  (TSKSUP). 


|LOF  Calculation  for  Number  of  Tasks  Suspended:  I 

mmii 

■1 

The  GLM  Procedure 

Dependent 

Variable : 

tsksup 

Sum  of 

Source 

DF 

Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

8 

1 ,79978182 

0.22497273 

22.23 

<,0001 

Error 

10 

0.10121818 

0.01012182 

Corrected  Total 

18 

1.901 

Source 

DF 

Type  III  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

0.0435125 

0.0435125 

4.3 

0.0649 

drd 

1 

0.3486125 

0.3486125 

34.44 

0.0002 

eel 

1 

0.0276125 

0.0276125 

2.73 

0.1296 

usp 

1 

0.1225125 

0.1225125 

12.1 

0.0059 

mrs 

1 

0.0946125 

0.0946125 

9.35 

0.0121 

Sum(IV’s)  = 

0.6368625 

3 

1.16291932 

0.38763977 

38.297 

Decision  Rule: 

Ho:  Oa'  =  0 

H,: 

a=  .05 

D.R.:  I  reject  Ho  if  1  Fobserved  ^  Ftabledl 

Fobsetved-LOF-TSKSUP  “ 

38.297 

*  Ftabled-LOF"=  ^(3,10)  ” 

6.55 

*  Winer,  p.  968  @  .01  level  (l-alpha=.99) 

.•.  Reject  Ho  with  the  result  that  LOF  for  TS] 

K.SUP  insignificant. 

LOF  For  Dependent  Variable  Number  of  Task  Drops  (TSKDRP). 


|LOF  Calculation  for  Number  of  Tasks  Dropped:  I 

The  GLM  Procedure 

Dependent 

Variable ; 

tskdrp 

Sum  of 

Source 

DF 

Squares 

Mean  Square 

F  Value 

Pr  >  F 

Model 

8 

39.19260096 

4.89907512 

80.57 

<.0001 

Error 

10 

0.60807273 

0.06080727 

Corrected  Total 

18 

39.80067368 

Source 

DF 

Type  III  SS 

Mean  Square 

F  Value 

Pr  >  F 

car 

1 

8.94645 

8.94645 

147.13 

A 

b 

o 

o 

drd 

1 

2.06045 

2.06045 

33.88 

0.0002 

eel 

1 

2.06045 

2.06045 

33.88 

0.0002 

usp 

1 

24.29045 

24.29045 

399.47 

b 

s 

V 

mrs 

1 

0.09245 

0.09245 

1.52 

0.2458 

Sum(IV's)  = 

37.45025 

LOF  =  SS(model)  -  Sum  SS(IV's)  = 

3 

1.74235096 

0.58078365 

9.5512 

Decision  Rule: 

a 

II 

O 

H,: 

o 

II 

a 

D.R..  I  reject  Hq  if  |  F^bservedl  ^  ^tabled 

Fobserved-LOF-TSKDRP  “ 

9.5512 

*  Ftabled-LOF~  ^(3,10)  = 

6.55 

*  Winer,  p.  968  @  .01  level  (l-alpha=.99) 

/.  Reject  Hq  with  the  result  that  LOF  for  TS] 

KDRP  ^significant. 

Regression  ANOVA  For  Dependent  Variable  Taskload  from  Task  Time  (TLTT). 

From  the  Regression  Model  ANOVA  svimmary  table  design  presented  in  Table  7,  the 
results  of  statistical  tests  performed  to  establish  the  significance  of  each  dependent  variable  to 
each  independent  variable  is  presented  in  Regression  ANOVA  tables.  The  determination  of 
significance  for  LOF  is  determined  by  a  manual  evaluation  using  a  decision  rule  and  comparing 
a  tabled  F  ratio  to  the  observed  F  ratio  as  calculated. 


Regression  ANOVA  Summary  Table  I 

Augmented  1-4  Replicate  2-5  Design 

Alternative  Regression  ANOVA 

Number  of  center  point  treatments,  n(cp)  = 

11 

Niunber  of  Data  Points  That  Have  Repeated  Observations 

(T)  (i.e.,  the  replication  of  the  Center  Point)  = 

1 

Treatment  Com 

>inations,  N  (n(l/4)  +  n(cp)  x  T)  = 

19 

Source 

df 

MS 

F 

P 

Regression 

(5) 

CAR 

1 

0.010 

0.160 

0.696 

DRD 

1 

0.877 

13.590 

0.004 

EEI 

1 

0.354 

5.480 

0.041 

USP 

1 

3.276 

50.730 

<.0001 

MRS 

1 

0.856 

13.260 

0.005 

Residuals 

(13) 

Lack  Of  Fit  (LOF) 

3 

0.901 

13.952 

Error  ((n(cp)  -1)  x  T) 

10 

0.065 

Total: 

(18) 

Verification:  N-1  = 

18 

'*  From  manual  decision  rule  test. 
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Regression  ANOVA  For  Dependent  Variable  Taskload  from  KSA  (TLKSA). 


Regression  ANOVA  Summary  Table  I 

Augmented  M  Replicate  2-5  Design 

Alternative  Regression  ANOVA 

Number  of  center  point  treatments,  n(cp)  = 

11 

Number  of  Data  Points  That  Have  Repeated  Observations 

(T)  (i.e.,  the  replication  of  the  Center  Point)  = 

1 

Treatment  Com 

binations,  N  (n(l/4)  +  n(cp)  x  T)  = 

19 

Source 

df 

E 

P 

Regression 

(5) 

CAR 

1 

0.412 

6.380 

0.030 

DRD 

1 

1.219 

18.880 

0.002 

EEI 

1 

0.158 

2.450 

0.149 

USP 

1 

368.955 

5714.540 

<.0001 

MRS 

1 

1.238 

19.170 

0.001 

Residuals 

(13) 

Lack  Of  Fit  (LOF) 

3 

1.075 

16.644 

"'Significant 

Error  ((n(cp)  -1)  x  T) 

10 

0.065 

Total: 

(18) 

Verification:  N-1  = 

18 

♦  From  manual  decision  rule  test. 
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Regression  ANOVA  For  Dependent  Variable  Utilization  (UTIL). 


Regression  ANOVA  Summary  T able  | 

Augmented  1-4  Replicate  2-5  Design 

Alternative  Regression  ANOVA 

Number  of  center  point  treatments,  n(cp)  = 

11 

Number  of  Data  Points  That  Have  Repeated  Observations 

(T)  (i.e.,  the  replication  of  the  Center  Point)  = 

1 

Treatment  Com 

binations,  N  (n(l/4)  +  n(cp)  x  T)  = 

19 

Source 

df 

MS 

F 

P 

Regression 

(5) 

CAR 

1 

0.00001013 

75.050 

<.0001 

DRD 

1 

0.00000841 

62.300 

<.0001 

EEI 

1 

0.00000050 

3.710 

0.083 

USP 

1 

0.00001624 

120.410 

<.0001 

MRS 

1 

0.00000338 

25.050 

0.001 

Residuals 

(13) 

Lack  Of  Fit  (LOF) 

3 

0.00000167 

12.872 

wm  1  mill  iiiinii 

10 

0.00004503 

Total: 

(18) 

Verification:  N-1  = 

18 

'*  From  manual  decision  rule  test. 


Regression  ANOVA  For  IDependent  Variable  Number  of  Queues  (NOQUE). 


Regression  ANOVA  Summary  Table  I 

Augmented  1-4  Replicate  2-5  Design 

Alternative  Regression  ANOVA 

Number  of  center  point  treatments,  n(cp)  = 

11 

Number  of  Data  Points  That  Have  Repeated  Observations 

(T)  (i.e.,  the  replication  of  the  Center  Point)  = 

1 

Treatment  ComI 

binations,  N  (n(l/4)  +  n(cp)  x  T)  = 

19 

Source 

df 

MS 

F 

P 

Regression 

(5) 

CAR 

1 

69.620 

259.290 

<.0001 

DRD 

1 

1.620 

6.030 

0.034 

EEI 

1 

19.220 

71.580 

<.0001 

USP 

1 

70.448 

262.370 

<.0001 

MRS 

1 

1.264 

4.710 

0.055 

Residuals 

(13) 

Lack  Of  Fit  (LOF) 

3 

0.741 

2.761 

Error  ((n(cp)  -1)  x  T) 

10 

2.685 

Total: 

(18) 

Verification:  N-1  = 

18 

*  From  manual  decision  rule  test. 


Regression  ANOVA  For  Dependent  Variable  Number  of  Task  Interrupts  (TSKINT). 


Regression  ANOVA  Summary  Table 

Augmented  1-4  Replicate  2-5  Design 

Alternative  Regression  ANOVA 

Number  of  center  point  treatments,  n(cp)  = 

11 

Number  of  Data  Points  That  Have  Repeated  Observations 

(T)  (i.e.,  the  replication  of  the  Center  Point)  = 

1 

Treatment  Com 

rinations,  N  (n(l/4)  +  n(cp)  x  T)  = 

19 

Source 

df 

MS 

F 

P 

Regression 

(5) 

CAR 

1 

0.162 

1.860 

0.202 

DRD 

1 

0.756 

8.670 

0.015 

EEI 

1 

1.674 

19.190 

0.001 

USP 

1 

6.588 

75.500 

<.0001 

MRS 

1 

0.162 

1.860 

0.202 

Residuals 

(13) 

Lack  Of  Fit  (LOF) 

3 

1.028 

11.782 

"'Significant 

Error  ((n(cp)  -1)  x  T) 

10 

0.873 

Total: 

(18) 

Verification:  N-1  = 

18 

From  manual  decision  rule  test. 


Regression  ANOVA  For  Dependent  Variable  Number  of  Task  Suspensions  (TSKSUP). 


Regression  ANOVA  Summary  Table  | 

Augmented  1-4  Replicate  2-5  Design 

Alternative  Regression  ANOVA 

Number  of  center  point  treatments,  n(cp)  = 

11 

Number  of  Data  Points  That  Have  Repeated  Observations 

(T)  (i.e.,  the  replication  of  the  Center  Point)  = 

1 

Treatment  Com 

binations,  N  (n(l/4)  +  n(cp)  x  T)  = 

19 

Source 

df 

MS 

F 

P 

Regression 

(5) 

CAR 

1 

0.044 

4.300 

0.065 

DRD 

1 

0.349 

34.440 

0.000 

EEI 

1 

0.028 

2.730 

0.130 

USP 

1 

0.123 

12.100 

0.006 

MRS 

1 

0.095 

9.350 

0.012 

Residuals 

(13) 

Lack  Of  Fit  (LOF) 

3 

0.388 

38.297 

Error  ((n(cp)  -1)  x  T) 

10 

0.010 

Total: 

(18) 

Verification:  N-1  = 

18 

*  From  manxial  decision  rule  test. 


Regression  ANOVA  For  Dependent  Variable  Number  of  Task  Drops  (TSKDRP). 


Regression  ANOVA  Summary  Table  | 

Augmented  1-4  Replicate  2-5  Design 

Alternative  Regression  ANOVA 

Number  of  center  point  treatments,  n(cp)  = 

11 

Number  of  Data  Points  That  Have  Repeated  Observations 

(T)  (i.e.,  the  replication  of  the  Center  Point)  = 

1 

Treatment  Com 

binations,  N  (n(l/4)  +  n(cp)xT)  = 

19 

Source 

df 

MS 

F 

P 

Regression 

(5) 

CAR 

1 

8.946 

147.130 

<.0001 

DRD 

1 

2.060 

33.880 

0.000 

EEI 

1 

2.060 

33.880 

0.000 

USP 

1 

24.290 

399.470 

<.0001 

MRS 

1 

0.092 

1.520 

0.246 

Residuals 

(13) 

Lack  Of  Fit  (LOF) 

3 

0.581 

9.551 

K311f}!TiRgWi 

Error  ((n(cp)  -1)  x  T) 

10 

0.061 

Total: 

(18) 

Verification:  N-1  = 

18 

*  From  manual  decision  rule  test. 


Appendix  J  -  Dependent  Variable  Response  Plots 
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Dependent  Variable  Taskload  From  Tasktime  -  TLTT. 


tltt/CAR 

Average  of  Sum  of -1  data 

-1 

-1.65591 

Average  of  Sum  of  0  data 

0 

0.26333 

Average  of  Sum  of  1  data 

1 

-1.24072 

tltt/DRD 

Average  of  Sum  of  -1  data 

-1 

-3.35114 

Average  of  Sum  of  0  data 

0 

0.26333 

Average  of  Sum  of  1  data 

1 

0.45451 

tltt/EEI 

Average  of  Sum  of  -1  data 

T 

-0.23939 

Average  of  Sum  of  0  data 

0 

0.26333 

Average  of  Sum  of  1  data 

-2.65725 

tltt/USP 

Average  of  Sum  of -1  data 

T 

2.22875 

Average  of  Sum  of  0  data 

0 

0.26333 

Average  of  Sum  of  1  data 

1 

-5.12538 

X 


tItt/CAR 


0.50000 

0.00000 

-0.50000 

-1.00000 

-1.50000 

-2.00000 

-2.50000 

-3.00000 


tltt/EEl 


tItt/USP 


From  these  curves  it  is  observed  that  all  four  independent  variables,  CAR,  DRD,  EEI,  and  USP 
invoke  a  quadratic  response  from  the  dependent  variable  TLTT, 
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Dependent  Variable  Taskload  From  KSA  -  TLKSA. 


tlksa/CAR 

Average  of  Sum  of  -1  data 

-1 

-0.00504 

Average  of  Sum  of  0  data 

0 

0.03703 

Average  of  Sum  of  1  data 

1 

-0.40224 

tlksa/DRD 

Average  of  Sum  of -1  data 

-1 

0.13808 

Average  of  Sum  of  0  data 

0 

0.03703 

Average  of  Sum  of  1  data 

1 

-0.54536 

tlksa/EEI 

Average  of  Sum  of -1  data 

-1 

-0.32674 

Average  of  Sum  of  0  data 

0 

0.03703 

Average  of  Sum  of  1  data 

1 

-0.08054 

tlksa/USP 

Average  of  Sum  of -1  data 

-1 

-6.14834 

Average  of  Sum  of  0  data 

0 

0.03703 

Average  of  Sum  of  1  data 

1 

5.74106 

tIksa/CAR 


tIksa/ORD 


% 


tIksa/USP 


8.00000 

6.00000 

4.00000 

2.00000 

0.00000 

-2.00000 

-4.00000 

•6.00000 

-8.00000 


From  these  curves  it  is  observed  that  three  independent  variables,  CAR,  DRD  and  EEI  invoke  a 
quadratic  response  from  the  dependent  variable  TLKSA  and  USP  is  linear. 
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Dependent  Variable  Utilization  -  UTIL. 


util/CAR 

Average  of  Sum  of -1  data 

-1 

1,62394 

Average  of  Sum  of  0  data 

0 

0.22205 

Average  of  Sum  of  1  data 

1 

-4.06650 

util/DRD 

Average  of  Sum  of -1  data 

-1 

1.37103 

Average  of  Sum  of  0  data 

0 

0.22205 

Average  of  Sum  of  1  data 

1 

-3.81359 

util/EEI 

Average  of  Sum  of  -1  data 

T 

-0.58901 

Average  of  Sum  of  0  data 

0 

0.22205 

Average  of  Sum  of  1  data 

-1.85355 

util/USP 

Average  of  Sum  of -1  data 

T 

2.38266 

Average  of  Sum  of  0  data 

0 

0.22205 

Average  of  Sum  of  1  data 

1 

-4.82522 

util/CAR 


util/DRD 


2.00000 

1.00000 

0.00000 

-1.00000 

-2.00000 

-3.00000 

-4.00000 

-5.00000 


2.00000 

1.00000 

0.00000 

-1.00000 

-2.00000 

-3.00000 

■4.00000 

-5.00000 


util/USP 


From  these  curves  it  is  observed  that  all  four  independent  variables,  CAR,  DRD,  EEI,  and  USP 
invoke  a  quadratic  response  from  the  dependent  variable  UTIL. 
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Dependent  Variable  Number  of  Queues  -  NOQUE. 


t 


I 

% 


noque/CAR 

Average  of  Sum  of  - 1  data 

-1 

-3.52635 

Average  of  Sum  of  0  data 

0 

-0.06304 

Average  of  Sum  of  1  data 

1 

4.21977 

noque/DRD 

Average  of  Sum  of -1  data 

-1 

-0.24410 

Average  of  Sum  of  0  data 

0 

-0.06304 

Average  of  Sum  of  1  data 

1 

0.93751 

noque/EEI 

Average  of  Sum  of -1  data 

T 

-1.68829 

Average  of  Sum  of  0  data 

0 

-0.06304 

Average  of  Sum  of  1  data 

JL 

2.38171 

noque/USP 

Average  of  Sum  of  - 1  data 

T 

-3.54932 

Average  of  Sum  of  0  data 

0 

-0.06304 

Average  of  Sum  of  1  data 

1 

4.24274 

noque/CAR 


noque/DRD 


1.00000 

0.80000 

0.60000 

0.40000 

0.20000 

0.00000 

-0.20000 

-0.40000 


noque/EEl 


noque/USP 


From  these  curves  it  is  observed  that  the  independent  variable  DRD  invokes  a  quadratic  response 
from  the  dependent  variable  NOQUE  Variables  CAR,  EEI,  and  USP  are  linear. 
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Dependent  Variable  Number  of  Task  Interrupts  -  TSKINT. 


tskint/CAR 

Average  of  Sum  of  - 1  data 

-1 

-1.27595 

Average  of  Sum  of  0  data 

0 

0.35255 

Average  of  Sum  of  1  data 

1 

-2.60211 

tskint/DRD 

Average  of  Sum  of -1  data 

.1 

-0.50818 

Average  of  Sum  of  0  data 

0 

0.35255 

Average  of  Sum  of  1  data 

1 

-3.36988 

tskint/EEI 

Average  of  Sum  of -1  data 

T 

0.18980 

Average  of  Sum  of  0  data 

0 

0.35255 

Average  of  Sum  of  1  data 

j_ 

-4.06786 

tskint/USP 

Average  of  Sum  of -1  data 

'a 

2.28373 

Average  of  Sum  of  0  data 

0 

0,35255 

Average  of  Sum  of  1  data 

1 

-6.16179 

4 


tskint/CAR 


tskint/DRD 


1.00000 

0.50000 

0.00000 

-0.50000 

-1.00000 

-1.50000 

-2.00000 

-2.50000 

-3.00000 
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From  these  curves  it  is  observed  that  all  four  independent  variables,  CAR,  DRD,  EEI,  and  USP 
invoke  a  quadratic  response  from  the  dependent  variable  TSKINT. 
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Dqiendent  Variable  Number  of  Task  Suspensions  -  TSKSUP. 
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From  these  curves  it  is  observed  that  all  four  independent  variables,  CAR,  DRD,  EEI,  and  USP 
invoke  a  quadratic  response  from  the  dependent  variable  TSKSUP. 


Dependent  Variable  Number  of  Task  Drops  -  TSKDRP. 
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From  these  curves  it  is  observed  that  all  four  independent  variables,  CAR,  DRD,  EEI,  and  USP 
invoke  a  quadratic  response  from  the  dependent  variable  TSKDRP. 
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Item  13  (continued) 


The  potential  specific  benefit  of  this  research  to  the  Army  is  twofold.  First,  the 
research  demonstrates  techniques  and  procedures  that  can  be  used  without  any 
required  modifications  to  the  existing  computer  simulations  that  allow  significant 
predictive  use  to  be  made  of  the  simulation  beyond  its  original  purpose  and  intent. 
Second,  the  use  of  these  techniques  with  CoHOST  is  developing  conclusions  and 
recommendations  from  that  simulation  that  Army  force  developers  can  use  with 
their  continuing  efforts  to  improve  and  enhance  the  ability  of  commanders  and 
other  decision  makers  to  perform  as  new  digital  communications  systems  and 
procedures  are  producing  radical  changes  to  the  paradigm  that  describes  the 
command  and  control  work  domain. 

I 

The  general  benefits  beyond  the  Army  domain  of  this  research  fall  into  the  two 
areas  of  methodological  improvement  of  simulation  based  experimental 
procedures  and  in  the  actual  application  area  of  the  CoHOST  simulation. 
Tailoring  the  experimental  controls  and  development  of  interrogation  techniques 
for  the  self-reporting  and  analysis  of  simulation  parameters  and  thresholds  are 
topics  that  bode  for  future  study.  The  CoHOST  simulation,  while  used  in  this 
thesis  as  an  example  of  new  and  tailored  techniques  for  computer  simulation 
based  research,  has  nevertheless  produced  conclusions  that  deviate  somewhat 
from  prevailing  thought  in  military  command  and  control.  Refinement  of  this 
simulation  and  its  use  in  an  even  more  thorough  simulation  based  study  could 
further  address  whether  the  military  decision  making  process  itself  or  contributing 
factors  such  as  development  of  mental  models  for  understanding  of  the  situation  is 
or  should  be  the  primary  focus  of  team  decision  makers  in  the  military  command 
and  control  domain.  r 


